Method of sharing and using sensor data

ABSTRACT

Sharing sensor data between a first device and a second device comprising obtaining a set of point data from at least one of sensors located in the first device, determining a property data of the subset of point data based on the subset of point data, generating a first sharing data for sharing with the second device based on the property data, transmitting the sharing data to the second device, identifying an occurrence of an event at a first time point, and generating a second sharing data different from the first sharing data. A content of the second sharing data includes at least a portion of the set of point data obtained within a first time period including the first time point.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and the benefit of U.S. PatentApplication No. 63/044,756, filed on Jun. 26, 2020, Korean PatentApplication No. 10-2020-0112084, filed on Sep. 3, 2020, Korean PatentApplication No. 10-2020-0112085, filed on Sep. 3, 2020, Korean PatentApplication No. 10-2020-0112086, filed on Sep. 3, 2020 the disclosure ofwhich is incorporated herein by reference in its entirety.

BACKGROUND 1. Field of the Invention

The present invention relates to a method of sharing and using sensordata acquired from a sensor, and more particularly, to a method ofdetermining the content of sharing data, which is shared depending onthe class of an object or depending on whether an event has occurred,and allowing a device that receives the sharing data through acommunication system to align a coordinate system of data included inthe sharing data to calculate a path of a vehicle using the sharingdata.

2. Discussion of Related Art

A sensor that acquires information on surrounding objects is anessential component in an autonomous driving system. For example, avehicle equipped with an autonomous driving system acquires informationon the surrounding environment of the vehicle on the basis of sensordata acquired through at least one of a light detection and ranging(LiDAR), a radar, a camera, an ultrasonic sensor, or the like.

Also, research on a vehicle-to-Everything (V2X)-based autonomous drivingsystem, which shares data with other vehicles, infrastructure devices,servers, and the like, is being actively conducted in order to improvethe quality of information acquired through limited sensors in theautonomous driving system.

SUMMARY OF THE INVENTION

An object to be achieved according to an embodiment is directed toproviding to a sensor data sharing method for the protection of personalinformation (private life).

An object to be achieved according to another embodiment is directed toproviding an efficient sensor data sharing method for the generation ofa high-definition map.

An object to be achieved according to still another embodiment isdirected to providing a selective sensor data sharing method accordingto event occurrence in order to increase data sharing efficiency.

An object to be achieved according to yet still another embodiment isdirected to providing a sensor data processing and vehicle controlmethod that prepares for risks (e.g., blind spots) that may occur when avehicle is being driven.

According to one embodiment, there is provided a method of sharingsensor data of a first device with a second device, the method includingobtaining, by a controller of the first device, a set of point data fromat least one of a sensors located in the first device, wherein the setof point data includes a first subset of point data representing atleast a portion of a first object, generating, by the controller, afirst property data of the first subset of point data based on the firstsubset of point data, wherein the first property data includes a classinformation of the first subset of point data, generating, by thecontroller, a sharing data including at least a portion of the firstsubset of point data and the first property data; and transmitting, bythe controller, the sharing data to the second device; wherein if aclass of a first object included in the class information a class inwhich personal information must be protected, a content of the sharingdata includes a privacy protection data in which at least a portion ofthe first subset of point data is processed such that personalinformation of the first object does not identified by the seconddevice.

According to another embodiment, there is provided a method of sharingsensor data of a first device with a second device, the method includingobtaining, by a controller of the first device, a set of point data fromat least one of a sensors located in the first device, wherein the setof point data includes a first subset of point data representing atleast a portion of a first object, generating, by the controller, afirst property data of the first subset of point data based on the firstsubset of point data, wherein the first property data includes a classinformation of the first subset of point data, generating a sharing datafor sharing with the second device using at least one of the firstsubset of point data and the first property data; and wherein whether acontent of the sharing data for sharing with the second device includesat least one of the first subset of point data or the first propertydata is determined based on at least one of a movability of a the firstobject's class and a type of the first object's class.

According to still another embodiment, there is provided a method ofsharing sensor data between a first device and a second device, themethod including obtaining, by a controller included in the firstdevice, a set of point data from at least one of a sensors located inthe first device, wherein the set of point data includes a plurality ofsubset of point data, determining, by the controller, a property data ofthe subset of point data based on the subset of point data, generating,by the controller, a first sharing data for sharing with the seconddevice based on the property data, transmitting, by the controller, thesharing data to the second device, wherein a content of the sharing dataincludes at least one of a plurality of pieces of information includedin the property data, identifying, by the controller, an occurrence ofan event at a first time point and generating, by the controller,according to identifying the event, a second sharing data different fromthe first sharing data, wherein a content of the second sharing dataincludes at least a portion of the set of point data obtained within afirst time period including the first time point.

According to still another embodiment, there is provided a method ofsharing sensor data between a first device and a second device, themethod including obtaining, by a controller included in the firstdevice, a set of point data included in a sensor data from at least oneof a sensors, wherein the set of point data includes subset of pointdata representing at least a portion of an object determining, by thecontroller, a property data of the subset of point data based on thesubset of point data, generating, by the controller, a first sharingdata for sharing with the second device based on the property data,transmitting, by the controller, the first sharing data to the seconddevice, wherein a content of the first sharing data includes at leastone of a plurality of pieces of information included in the propertydata to the second device, identifying, by the controller, occurrence ofan event at a first time point and generating, by the controller,according to identifying the event, a second sharing data different fromthe first sharing data, wherein a content of the second sharing dataincludes at least a portion of the set of point data obtained within afirst time period including the first time point.

According to still another embodiment, there is provided a method ofworking of a server, the method including identifying an event occurredin a first region at a first time, transmitting a first message torequest a sensor data to a first device located within a first rangefrom the first region, wherein the first message includes a timeinformation of the event, wherein the time information is related to thefirst time in order to obtain the sensor data obtained within a timeperiod related to the first time, transmitting a second message tonotify the event to a second device located within a second rangerepresenting a predetermined region outside the first range, wherein thesecond message includes a location information of the event, wherein thelocation information is related to the first region such that the eventis identified by the second device and receiving at least a portion ofset of point data obtained within a first time period including thefirst time in response to the first message, wherein the set of pointdata is obtained from at least one of sensors located in the firstdevice.

According to still another aspect, there is provided a method ofprocessing sharing data to control a vehicle, the method includingobtaining, by a controller included in the vehicle, a first set of pointdata included in sensor data acquired from a first sensor included inthe vehicle, wherein the first set of point data includes a first subsetof point data representing at least a portion of a first object,acquiring, by the controller, first property data of the first subset ofpoint data corresponding to a position of the first object, wherein thefirst property data is shown by a first coordinate system based on afirst origin, generating, by the controller, first standard propertydata on the basis of the first property data, wherein the first standardproperty data is shown in a second coordinate system based on a secondorigin, acquiring, by the controller, second standard property datacorresponding to a position of a second object not represented by thefirst set of point data, wherein the second standard property data isshown in the second coordinate system, and controlling, by thecontroller, the vehicle on the basis of the first standard property dataand the second standard property data, wherein the second standardproperty data is generated based on second property data of a secondsubset of point data included in a second set of point data, and whereinthe second set of point data is acquired from a second sensor includedin the first device.

According to still another aspect, there is provided a method ofgenerating a path of a vehicle, the method including acquiring, by acontroller included in the vehicle, a first set of point data includedin sensor data acquired from a first sensor included in the vehicle,wherein the first set of point data includes a first subset of pointdata representing at least a portion of a first object, determining, bythe controller, first property data of the first subset of point data,wherein the first property data corresponds to the first object,generating, by the controller, a local path of the vehicle on the basisof at least one of the first set of point data or the first propertydata, wherein the local path of the vehicle includes at least one of aspeed of the vehicle, a direction of the vehicle, and a position of thevehicle, receiving, by the controller, second property data determinedbased on second set of point data included in sensor data acquired froma second sensor placed in a first device, wherein the second propertydata corresponds to a second object that is not recognized based on thefirst set of point data, and generating, by the controller, a modifiedpath by changing at least some of the position of the vehicle, the speedof the vehicle, or the direction of the vehicle in the local path of thevehicle on the basis of the second property data and at least one of thefirst set of point data, the first property data, or the local path.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become more apparent to those of ordinary skill in theart by describing exemplary embodiments thereof in detail with referenceto the accompanying drawings, in which:

FIG. 1 is a diagram illustrating elements of an autonomous drivingsystem according to an embodiment.

FIG. 2 is a diagram specifically illustrating at least one sensoraccording to an embodiment.

FIG. 3 is a diagram showing a display scheme through an infotainmentsystem according to an embodiment.

FIG. 4 is a diagram showing a situation in which an autonomous drivingsystem detects a moving object showing an abnormal driving patternaccording to an embodiment.

FIG. 5 is a diagram showing a situation in which an autonomous drivingsystem recognizes an accident of a vehicle in front while drivingaccording to an embodiment.

FIG. 6 is a diagram showing a situation in which an autonomous drivingsystem recognizes a sudden backward movement of a vehicle in frontaccording to an embodiment.

FIG. 7 is a diagram showing a situation in which an autonomous drivingsystem tracks the movement of a vehicle's wheel according to anembodiment.

FIG. 8 is a diagram illustrating a method of detecting, by an autonomousdriving system, black ice present on a road according to an embodiment.

FIG. 9 is a diagram showing a situation in which a vehicle equipped withan autonomous driving system detects an illegally parked vehicle whiledriving according to an embodiment.

FIG. 10 is a diagram showing a situation in which an autonomous drivingsystem detects an available parking space according to an embodiment.

FIG. 11 is a diagram showing a process of generating, by an autonomousdriving system, a map for pulling out a vehicle according to anembodiment.

FIG. 12 is a diagram illustrating types of a communication systemaccording to an embodiment.

FIG. 13 is a diagram showing a situation in which a traffic event hasoccurred in front of a vehicle equipped with an autonomous drivingsystem according to an embodiment.

FIG. 14 is a diagram showing a situation in which a vehicle equippedwith an autonomous driving system recognizes an available parking spacethrough communication with an infrastructure device in a parking lotaccording to an embodiment.

FIG. 15 is a diagram showing a situation in which a vehicle equippedwith an autonomous driving system acquires sensor data regarding anenvironment around the vehicle through at least one sensor according toan embodiment.

FIG. 16 is a diagram showing, on a three-dimensional (3D) map, sensordata acquired by a light detection and ranging (LiDAR) device placed inthe vehicle of FIG. 15 ;

FIG. 17 is a diagram schematically showing sensor data included in the3D map of FIG. 16 in a two-dimensional (2D) plane;

FIG. 18 is a diagram illustrating point data acquired from at least oneLiDAR device included in an autonomous driving system according to anembodiment.

FIG. 19 is a diagram illustrating a set of point data acquired from aLiDAR device included in the vehicle of FIG. 16 ;

FIG. 20 is a diagram illustrating a subset of point data acquired fromat least one LiDAR device included in an autonomous driving systemaccording to an embodiment.

FIG. 21 is a diagram illustrating property data generated from a subsetof point data acquired from a LiDAR device included in an autonomousdriving system according to an embodiment.

FIG. 22 is a diagram showing another example of the property data ofFIG. 21 ;

FIG. 23 is a diagram illustrating a plurality of pieces of informationincluded in property data according to an embodiment.

FIG. 24 is a diagram illustrating class information included in propertydata according to an embodiment.

FIG. 25 is a diagram illustrating center position information includedin property data according to an embodiment.

FIG. 26 is a diagram illustrating size information included in propertydata according to an embodiment.

FIG. 27 is a diagram illustrating template information of shapeinformation included in property data according to an embodiment.

FIG. 28 is a diagram illustrating skeleton information of shapeinformation included in property data according to an embodiment.

FIG. 29 is a diagram showing that an autonomous driving system matches asubset of point data acquired from a sensor to a high-definition mapaccording to an embodiment.

FIG. 30 is a diagram showing that an autonomous driving system matchesproperty data of an object to a high-definition map according to anembodiment.

FIG. 31 is a diagram showing a situation in which an autonomous drivingsystem changes a path to avoid an obstacle obstructing the driving of avehicle according to an embodiment.

FIG. 32 is a diagram showing a situation in which data is shared betweena plurality of devices according to an embodiment.

FIG. 33 is a diagram showing the types of content of sharing data thatmay be included in sharing data according to an embodiment.

FIG. 34 is a diagram specifically showing the content of sharing data ofFIG. 33 ;

FIG. 35 is a diagram showing a situation in which sensor data is sharedbetween a vehicle and an infrastructure device;

FIG. 36 is a diagram illustrating a situation in which a set of pointdata is included in the content of sharing data according to anembodiment.

FIG. 37 is a diagram illustrating a method of processing, by a firstvehicle, a shared first set of point data and a second set of point dataaccording to an embodiment.

FIG. 38 is a diagram illustrating a method of processing, by a firstvehicle, a shared first set of point data and a second set of point dataaccording to another embodiment.

FIG. 39 is a diagram illustrating a situation in which property data isincluded in the content of sharing data according to an embodiment.

FIG. 40 is a flowchart illustrating a selective sharing method of sensordata according to an embodiment.

FIG. 41 is a diagram showing a situation in which a first vehicleacquires sensor data to selectively share the sensor data according toan embodiment.

FIG. 42 is a diagram schematically representing, in a 2D plane, thesensor data acquired by the first vehicle through a LiDAR deviceaccording to FIG. 41 ;

FIG. 43 is a diagram showing class information and property data of aplurality of subsets of point data included in sensor data according toan embodiment.

FIG. 44 is a diagram showing the content of sharing data transmitted bya first vehicle according to an embodiment.

FIG. 45 is a diagram illustrating privacy protection data included inthe content of sharing data according to an embodiment.

FIG. 46 is a flowchart illustrating a method of selectively sharing datadepending on whether approval for data sharing is gained from anexternal server in a data sharing system according to an embodiment.

FIG. 47 is a flowchart illustrating a detailed method of selectivelysharing sensor data according to another embodiment.

FIG. 48 is a diagram showing a situation in which a first vehicleacquires sensor data to selectively share the sensor data according toan embodiment.

FIG. 49 is a diagram schematically representing sensor data acquired bythe first vehicle through a LiDAR device according to FIG. 48 in a 2Dplane;

FIG. 50 is a diagram illustrating the content of sharing data accordingto an embodiment.

FIG. 51 is a flowchart illustrating a method of selectively sharingsensor data including additional information according to an embodiment.

FIG. 52 is a diagram showing a situation in which a first vehicleacquires additional information through at least one sensor according toan embodiment.

FIG. 53 is a diagram schematically showing, in a 2D plane, the sensordata acquired by the first vehicle according to FIG. 52 ;

FIG. 54 is a diagram illustrating a subset of point data and additionalinformation included in the content of sharing data according to anembodiment.

FIG. 55 is a flowchart illustrating a method of sharing sensor datarelated to a movable object according to an embodiment.

FIG. 56 is a diagram illustrating a method of selectively storingsharing data according to an embodiment.

FIG. 57 is a flowchart illustrating a selective sharing method of sensordata according to another embodiment.

FIG. 58 is a diagram showing a situation in which a first vehicleacquires sensor data before an event occurs according to an embodiment.

FIG. 59 is a diagram schematically showing a set of point data includedin the sensor data acquired according to FIG. 58 in a 2D plane;

FIG. 60 is a diagram illustrating first sharing data transmitted by afirst vehicle before an event occurs according to an embodiment.

FIG. 61 is a diagram showing a situation in which a first vehicleacquires sensor data when an event occurs according to an embodiment.

FIG. 62 is a diagram schematically showing a set of point data includedin the sensor data acquired according to FIG. 61 in a 2D plane;

FIG. 63 is a diagram illustrating second sharing data transmitted by afirst vehicle after an event occurs according to an embodiment.

FIG. 64 is a diagram illustrating a situation in which a traffic eventhas occurred according to an embodiment.

FIG. 65 is a diagram illustrating a situation in which an environmentalevent has occurred according to an embodiment.

FIG. 66 is a diagram illustrating a situation in which a regulatoryevent has occurred according to an embodiment.

FIG. 67 is a diagram illustrating a method of requesting, by a server,data regarding a traffic event or indicating that a traffic event hasoccurred according to an embodiment.

FIG. 68 is a diagram showing a situation in which a server and a vehiclecommunicate with each other to share data according to an embodiment.

FIG. 69 is a diagram illustrating a first sub-range included in a firstrange according to an embodiment.

FIG. 70 is a diagram illustrating data included in the sharing datatransmitted by a first vehicle to a server according to an embodiment.

FIG. 71 is a diagram illustrating information included in a firstmessage according to an embodiment.

FIG. 72 is a diagram illustrating information included in a secondmessage according to an embodiment.

FIG. 73 is a diagram illustrating an example related to a selectivesharing method of sensor data depending on the range;

FIG. 74 is a diagram illustrating a selective data sharing methodaccording to a blind spot during the driving of a vehicle in relation toa regulatory event according to an embodiment.

FIG. 75 is a flowchart illustrating a scheme of processing property dataincluded in sharing data according to an embodiment.

FIG. 76 is a diagram showing a situation in which a vehicle and aninfrastructure device acquire sensor data to perform data sharingaccording to an embodiment.

FIG. 77 is a diagram illustrating a method in which a controller of avehicle shows first property data and first standard property data in afirst local coordinate system and a global coordinate system,respectively, according to an embodiment.

FIG. 78 is a diagram illustrating a method in which a controller of avehicle generates second standard property data on the basis of secondproperty data shown in a second local coordinate system according to anembodiment.

FIG. 79 is a diagram illustrating a global path according to anembodiment.

FIG. 80 is a diagram illustrating a local path and a modified pathaccording to an embodiment.

FIG. 81 is a flowchart illustrating a method of generating or modifying,by a vehicle, a path on the basis of sharing data according to anembodiment.

FIG. 82 is a diagram showing a situation in which a first vehicletravels along a path generated based on sensor data and sharing dataaccording to an embodiment.

FIG. 83 is a diagram illustrating a method of generating a modified pathon the basis of a collision probability map generated by a controller ofa first vehicle according to an embodiment.

FIG. 84 is a diagram illustrating various examples of a modified pathaccording to an embodiment.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments described in this specification are intended to clearlyexplain the spirit of the invention to those skilled in the art.Therefore, the present invention is not limited by the embodiments, andthe scope of the present invention should be interpreted as encompassingmodifications and variations without departing from the spirit of theinvention.

Terms used in this specification are selected from among general terms,which are currently widely used, in consideration of functions in thepresent invention and may have meanings varying depending on intentionsof those skilled in the art, customs in the field of the art, theemergence of new technologies, or the like. If a specific term is usedwith a specific meaning, the meaning of the term will be describedspecifically. Accordingly, the terms used in this specification shouldnot be defined as simple names of the components but should be definedon the basis of the actual meaning of the terms and the whole contextthroughout the present specification.

The accompanying drawings are to facilitate the explanation of thepresent invention, and the shape in the drawings may be exaggerated forthe purpose of convenience of explanation, so the present inventionshould not be limited by the drawings.

When it is determined that detailed descriptions of well-known elementsor functions related to the present invention may obscure the subjectmatter of the present invention, detailed descriptions thereof will beomitted herein as necessary.

According to one embodiment, there is provided a method of sharingsensor data of a first device with a second device, the method includingobtaining, by a controller of the first device, a set of point data fromat least one of a sensors located in the first device, wherein the setof point data includes a first subset of point data representing atleast a portion of a first object, generating, by the controller, afirst property data of the first subset of point data based on the firstsubset of point data, wherein the first property data includes a classinformation of the first subset of point data, generating, by thecontroller, a sharing data including at least a portion of the firstsubset of point data and the first property data; and transmitting, bythe controller, the sharing data to the second device; wherein if aclass of a first object included in the class information a class inwhich personal information must be protected, a content of the sharingdata includes a privacy protection data in which at least a portion ofthe first subset of point data is processed such that personalinformation of the first object does not identified by the seconddevice.

In some embodiments, the class in which personal information must beprotected includes one of a class related to human, a class related toidentification number of a vehicle or a building, or a class related toID.

In some embodiments, the class information of the first subset of pointdata includes at least one of a information about a type of the firstobject, a information about a type of a portion of the first object, ora information about a situation of a region related to the first object.

In some embodiments, the first property data of the first subset ofpoint data includes at least one of a class information of the firstobject, a center position information representing a center position ofthe first subset of point data, a size information representing a sizeof the first subset of point data, a movement information including atleast one of a velocity or a direction of the first subset of pointdata, or a shape information represented by processing the shape of thefirst object.

In some embodiments, the content of the sharing data includes at leastone of information included in the first property data regardless of atype of class included in the class information of the first subset ofpoint data.

In some embodiments, the shape information is determined based on theclass information off the first subset of point data, and wherein theshape information includes at least one of a skeleton informationindicated by points less than a predetermined number or at least one ofline, and template information in which the first object is representedin a predetermined shape.

In some embodiments, the privacy protection data includes at leastportion of information included in the first property data, and whereinthe first property data includes a shape information represented byprocessing the shape of the first object.

In some embodiments, the first subset of point data includes a multiplepieces of point data, and wherein the privacy protection data isgenerated based on at least one of the multiple pieces of point datacorresponding to a region related to privacy of the first object.

In some embodiments, the set of point data includes a second subset ofpoint data representing at least portion of a second object, and whereinif a class of the first object included in the class information of thefirst subset of point data is a class in which personal information mustbe protected and a class of the second object included in the classinformation of the second subset of point data is not a class in whichpersonal information must be protected, the content of the sharing dataincludes a privacy protection data in which at least a portion of thefirst subset of point data is processed, and the content of the sharingdata includes at least portion of a second property data of the secondsubset of point data.

In some embodiments, the set of point data includes a second subset ofpoint data representing at least portion of a second object, wherein ifobtaining a approval about sharing of at least one of the second subsetof point data or a second property data, and wherein the second propertydata includes a class information of the second subset of point data,the content of the sharing data includes at least one of the secondsubset of point data or the second property data regardless of a type ofthe second object's class included in the class information of thesecond subset of point data.

In some embodiments, the set of point data includes a multiple pieces ofpoint data, wherein the multiple pieces of point data generated based onat least one of a distance to an object measured by at least one of thesensors disposed to the first device or a reflectance of the object,wherein the controller generates the first subset of point data based onthe multiple pieces of point data of the first object which locatedwithin a predetermined distance from the first device, wherein thecontroller generates a second subset of point data based on the multiplepieces of the second object which located farther than the predetermineddistance from the first device, and wherein the content of the sharingdata includes the second subset of point data regardless of a propertydata of the second subset of point data.

According to another embodiment, there is provided a method of sharingsensor data of a first device with a second device, the method includingobtaining, by a controller of the first device, a set of point data fromat least one of a sensors located in the first device, wherein the setof point data includes a first subset of point data representing atleast a portion of a first object, generating, by the controller, afirst property data of the first subset of point data based on the firstsubset of point data, wherein the first property data includes a classinformation of the first subset of point data, generating a sharing datafor sharing with the second device using at least one of the firstsubset of point data and the first property data; and wherein whether acontent of the sharing data for sharing with the second device includesat least one of the first subset of point data or the first propertydata is determined based on at least one of a movability of a the firstobject's class and a type of the first object's class.

In some embodiments, if the first object's class included in the classinformation of the first subset of point data is related to an immovableobject, the content of the sharing data for sharing with the seconddevice includes the first subset of point data, and the method includestransmitting the sharing data to the second device.

In some embodiments, the content of the sharing data includes at leastone of a plurality of pieces of information included in the firstproperty data of the first subset of point data.

In some embodiments, if the controller obtains an additional informationrelated to whether the immovable object is movable after certain time,the content of the sharing data includes the additional information.

In some embodiments, if the first object's class included in the classinformation of the first subset of point data is related to a movableobject, the content of the sharing data does not include the firstsubset of point data.

In some embodiments, the content of the sharing data includes at leastone of a plurality of pieces of information included in the firstproperty data of the first subset of point data, the method includestransmitting the sharing data to the second device.

In some embodiments, the class information of the first subset of pointdata includes at least one of an information about a type of the firstobject, an information about a type of a portion of the first object, oran information about a situation of a region related to the firstobject.

In some embodiments, the set of point data includes a second subset ofpoint data representing at least portion of a second object, wherein thesecond object located in a region separated by a predetermined distancefrom the first object, wherein the situation of the region related tothe first object is determined based on the first subset of point dataand the second subset of point data, wherein if the class information ofthe first subset of point data and a class information of the secondsubset of point data include an information about the situation of theregion related to the first object, the controller obtains an additionalinformation related to an end time of the situation of the regionrelated to the first object, and wherein the content of the sharing dataincludes the additional information.

In some embodiments, if the first object's class included in the classinformation of the first subset of point data is related to an immovableobject, the content of the sharing data does not include the firstsubset of point data, and wherein if the first object's class includedin the class information of the first subset of point data is related toa movable object, the content of the sharing data includes the firstsubset of point data.

In some embodiments, at least one of the sensors includes at least oneof a LiDAR, a camera, a radar and an ultrasonic sensor.

In some embodiments, each of the first device and the second deviceincludes at least one of a moving object, a server, a mobile device, oran infrastructure device.

In some embodiments, there is provided a computer-readable recordingmedium having a program recorded thereon to perform the above-describedvehicle control method and path generation method.

According to still another embodiment, there is provided a method ofsharing sensor data between a first device and a second device, themethod including obtaining, by a controller included in the firstdevice, a set of point data from at least one of a sensors located inthe first device, wherein the set of point data includes a plurality ofsubset of point data, determining, by the controller, a property data ofthe subset of point data based on the subset of point data, generating,by the controller, a first sharing data for sharing with the seconddevice based on the property data, transmitting, by the controller, thesharing data to the second device, wherein a content of the sharing dataincludes at least one of a plurality of pieces of information includedin the property data, identifying, by the controller, an occurrence ofan event at a first time point and generating, by the controller,according to identifying the event, a second sharing data different fromthe first sharing data, and wherein a content of the second sharing dataincludes at least a portion of the set of point data obtained within afirst time period including the first time point.

In some embodiments, the method is configured to transmit the secondsharing data to the second device.

In some embodiments, if receiving a request information requesting toshare the second sharing data from at least one of the second device ora third device, in response to receiving the request information, themethod being configured to transmit the second sharing data to a devicetransmitting the request information.

In some embodiments, if receiving a request information from at leastone of the second device or a third device requesting to share thesecond sharing data to a fourth device, in response to receiving therequest information, the method being configured to transmit the secondsharing data to the fourth device.

In some embodiments, identifying the event comprise obtaining aninformation indicating the occurrence of the event from at least one ofthe second device or a third device.

In some embodiments, identifying the event comprise identifying theoccurrence of the event based on at least a portion of the set of pointdata, the plurality of subset of point data or the property data of thesubset of point data.

In some embodiments, the request information includes an informationindicating the occurrence of the event, and wherein identifying theevent comprise identifying the occurrence of the event based on theinformation indicating the occurrence of the event.

In some embodiments, one of the plurality of subset of point datarepresents at least a portion of an object related to the event.

In some embodiments, the event includes at least one of a traffic-eventrelated to at least one of accident related to the first device oraccident related to another device around the first device, anenvironment event related to environment around the first device, and aregulatory event related to regulatory about the first device or anotherdevice around the first device.

In some embodiments, the first time point includes at least one of atime point at which the event identified or a time point at which theevent occurred.

In some embodiments, a content of the second sharing data includes atleast a portion of the content of the first sharing data.

In some embodiments, the second sharing data is generated based on aplurality of set of point data obtained during the first time period,when the second sharing data is generated at regular intervals,transmitting the second sharing data to the second device whenever thesecond sharing data is generated, or when the second sharing data isgenerated after the end of the first time period, transmitting thesecond sharing data to the second device after the second sharing datais generated.

In some embodiments, the first time period includes a time point atwhich the event occurred.

In some embodiments, the first time period includes a second time pointat which the event ends.

According to still another embodiment, there is provided a method ofsharing sensor data between a first device and a second device, themethod including obtaining, by a controller included in the firstdevice, a set of point data included in a sensor data from at least oneof a sensors, wherein the set of point data includes subset of pointdata representing at least a portion of an object determining, by thecontroller, a property data of the subset of point data based on thesubset of point data, generating, by the controller, a first sharingdata for sharing with the second device based on the property data,transmitting, by the controller, the first sharing data to the seconddevice, wherein a content of the first sharing data includes at leastone of a plurality of pieces of information included in the propertydata to the second device, identifying, by the controller, occurrence ofan event at a first time point and generating, by the controller,according to identifying the event, a second sharing data different fromthe first sharing data, and wherein a content of the second sharing dataincludes at least a portion of the set of point data obtained within afirst time period including the first time point.

According to still another embodiment, there is provided a method ofworking of a server, the method including identifying an event occurredin a first region at a first time, transmitting a first message torequest a sensor data to a first device located within a first rangefrom the first region, wherein the first message includes a timeinformation of the event, wherein the time information is related to thefirst time in order to obtain the sensor data obtained within a timeperiod related to the first time, transmitting a second message tonotify the event to a second device located within a second rangerepresenting a predetermined region outside the first range, wherein thesecond message includes a location information of the event, wherein thelocation information is related to the first region such that the eventis identified by the second device and receiving at least a portion ofset of point data obtained within a first time period including thefirst time in response to the first message, and wherein the set ofpoint data is obtained from at least one of sensors located in the firstdevice.

In some embodiments, the event includes at least one of a traffic-eventrelated to at least one of accident related to the first device oraccident related to another device around the first device, anenvironment event related to environment around the first device, and aregulatory event related to regulatory about the first device or anotherdevice around the first device.

In some embodiments, when the first device is located in a first subrange, the set of point data obtained from at least one of sensorslocated in the first device includes a subset of point data representingat least a portion of an object related to the event, and wherein thefirst sub range represents an area in which information related to theevent can be obtained within the first range.

In some embodiments, the first region includes a region including all ofobjects related to the event.

In some embodiments, identifying the event comprise obtaining a firstinformation representing that the event occurs at the first time and asecond information representing that the event occurs in the firstregion.

In some embodiments, the second device is included in a vehicle, andwherein when a path of the vehicle located in the second range isrelated to the first region, transmitting the second message to thevehicle.

In some embodiments, each of the first device and the second deviceincludes at least one of a moving object, a server, a mobile device, oran infrastructure device.

In some embodiments, at least one of the sensors includes at least oneof a LiDAR, a camera, a radar and an ultrasonic sensor.

In some embodiments, there is provided a computer-readable recordingmedium having a program recorded thereon to perform the above-describedvehicle control method and path generation method.

According to still another embodiment, there is provided a method ofprocessing sensor data obtained from a first device to control avehicle, the method including obtaining, by a controller included in thevehicle, a first set of point data included in sensor data obtained froma first sensor included in the vehicle, wherein the first set of pointdata includes a first subset of point data representing at least aportion of a first object; obtaining, by the controller, a firstproperty data of the first subset of point data corresponding to aposition of the first object, wherein the first property data isrepresented by a first coordinate system based on a first origin;generating a first standard property data on the basis of the firstproperty data, wherein the first standard property data is representedby a second coordinate system based on a second origin; obtaining, bythe controller, a second standard property data corresponding to aposition of a second object not represented by the first set of pointdata, wherein the second standard property data is represented by thesecond coordinate system; and controlling, by the controller, thevehicle on the basis of the first standard property data and the secondstandard property data, wherein the second standard property data isgenerated based on a second property data of a second subset of pointdata included in a second set of point data, and wherein the second setof point data is obtained from a second sensor included in the firstdevice.

In some embodiments, the generating of the first standard property datamay include setting the first coordinate system in which the firstproperty data is represented as the second coordinate system.

In some embodiments, the obtaining of second standard property data mayinclude receiving the second property data represented by a thirdcoordinate system based on a third origin from the first device andgenerating the second standard property data on the basis of the secondproperty data by aligning the third coordinate system with the secondcoordinate system.

In some embodiments, the generating of the first standard property datamay include aligning the first coordinate system in which the firstproperty data is represented with the second coordinate system, and thegenerating the second standard property data may include aligning thethird coordinate system in which the second property data is representedwith the second coordinate system.

In some embodiments, the third origin may correspond to a position of anoptical origin of the second sensor included in the first device.

In some embodiments, the first origin may correspond to a position of anoptical origin of the first sensor included in the vehicle, and thesecond origin may correspond to at least one of the first origin or apredetermined static position.

In some embodiments, the first property data may include at least one ofa class information of the first object, a center position informationindicating a center position of the first subset of point data, a sizeinformation indicating a size information of the first subset of pointdata, a movement information including at least one of a movement speedor a movement direction of the first subset of point data, anidentification information for distinguishing the first subset of pointdata from other subsets of point data, and a shape information obtainedby processing a shape of the first object, and the second property datamay include at least one of a class information of the second object, acenter position information indicating a center position of the secondsubset of point data, a size information indicating a size informationof the second subset of point data, a movement information including atleast one of a movement speed or a movement direction of the secondsubset of point data, an identification information for distinguishingthe second subset of point data from other subsets of point data, and ashape information obtained by processing a shape of the second object.

In some embodiments, the first property data may include a first centerposition information of the first subset of point data represented bythe first coordinate system, a first standard center positioninformation included in the first standard property data and generatedbased on the first center position information is represented by thesecond coordinate system, the second property data includes secondcenter position information of the second subset of point datarepresented by a third coordinate system, and the second standard centerposition information included in the second standard property data andgenerated based on the second center position information may berepresented by the second coordinate system.

In some embodiments, the controlling of the vehicle may includecontrolling the vehicle to travel along a preset global path on thebasis of a position of the vehicle and a position of a destination andgenerating a local path on the basis of the first standard property dataand the second standard property data.

According to still another embodiment, there is provided a method ofprocessing sensor data obtained from a first device to generate a pathof a vehicle, the method including, by a controller included in thevehicle, obtaining a first set of point data included in sensor dataacquired from a first sensor included in the vehicle, wherein the firstset of point data includes a first subset of point data representing atleast a portion of a first object; determining, by the controller, afirst property data of the first subset of point data, wherein the firstproperty data corresponds to the first object; generating, by thecontroller, a local path of the vehicle on the basis of at least one ofthe first set of point data or the first property data, wherein thelocal path of the vehicle includes at least one of a speed of thevehicle, a direction of the vehicle, and a position of the vehicle;receiving, by the controller, a second property data determined based ona second set of point data included in sensor data acquired from asecond sensor placed in the first device, and wherein the secondproperty data corresponds to a second object that is not recognizedbased on the first set of point data; and generating, by the controller,a modified path by changing at least some of the position of thevehicle, the speed of the vehicle, or the direction of the vehicle inthe local path of the vehicle on the basis of the second property dataand at least one of the first set of point data, the first propertydata, or the local path.

In some embodiments, the local path may at least partially overlap acertain region where the second object is positioned, and the modifiedpath may not overlap the certain region where the second object ispositioned.

In some embodiments, the vehicle may be controlled to travel along apreset global path on the basis of a position of the vehicle and aposition of a destination, and the generating of the local path mayinclude generating a local path including at least a portion of a regioncorresponding to the field of view of the first sensor; and controllingthe vehicle to travel along the local path.

In some embodiments, the generating of the modified path may includedetermining whether to modify the path of the vehicle on the basis ofthe probability of movement of the vehicle predicted based on the localpath of the vehicle and the probability of movement of the second objectpredicted based on the second property data.

In some embodiments, the method may further include receiving a thirdproperty data determined based on the second set of point data acquiredfrom the second sensor placed in the first device, wherein the thirdproperty data corresponds to a third object; comparing the thirdproperty data and the first property data and determining whether thethird object and the first object are the same object, and generating amodified path for considering the third object by changing at least someof a position of the vehicle, a speed of the vehicle, or a direction ofthe vehicle on the basis of the third property data, the second propertydata, and at least one of the first set of point data, the firstproperty data, or the local path.

In some embodiments, the method may further include receiving a thirdproperty data determined based on the second set of point data acquiredfrom the second sensor placed in the first device, wherein the thirdproperty data corresponds to a third object; and comparing the thirdproperty data and the first property data and determining whether thethird object and the first object are the same object, and wherein whenit is determined that the first object and the third object are the sameobject, the controller does not generate the modified path forreflecting the third object.

In some embodiments, the modified path may include at least one of afirst modified path and a second modified path, the first modified pathmay include a path obtained by changing at least a portion of the localpath of the vehicle, and the second modified path may include a path forstopping the vehicle in the local path of the vehicle.

In some embodiments, the first device may include at least one of amoving object, an infrastructure, a mobile device, or a server.

In some embodiments, each of the first sensor and the second sensor mayinclude at least one of a LiDAR, a camera, a radar, and an ultrasonicsensor.

In some embodiments, there is provided a computer-readable recordingmedium having a program recorded thereon to perform the above-describedvehicle control method and path generation method.

1. Overview of Autonomous Driving System 1.1. Advanced Driver AssistanceSystems (ADAS)

Advanced driver-assistance systems, which are abbreviated as “ADAS,” aresystems that assist drivers in driving and may refer to systems that canreduce drivers' fatigue and help drivers to drive safely.

Advanced driver-assistance systems may include various devices andsystems.

For example, the advanced driver-assistance systems may include anautomatic vehicle navigation device, an adaptive cruise control device,a lane keeping assistance system, a lane departure prevention assistancesystem, a blind spot warning device, an intelligent speed adaptationsystem, an intelligent headlight control system, a pedestrian protectionsystem, an automatic parking system, a traffic sign recognition system,a driver drowsiness prevention system, a vehicle communication system, ahill descent control system, an electric vehicle driving warning system,a low-beam assistance system, a high-beam assistance system, a frontcollision warning system, smart cruise control (SCC), navigation-basedsmart cruise control (NSCC), a highway driving assistance system, a rearview monitor with e-Mirror (RVM), etc., but the present invention is notlimited thereto.

Also, a device equipped with the driver assistance system may share datawith other devices through communication. This will be described indetail below.

1.2. Autonomous Driving System (AD)

Also, an autonomous driving system (e.g., autonomous driving (AD),autonomous car, driverless car, self-driving car, robotic car) may bemounted in a vehicle to enable the vehicle to automatically travelwithout human intervention.

Also, the autonomous driving system may share data with other devicesthrough communication. This will be described in detail below

Hereinafter, for convenience of description, the above-described driverassistance system and autonomous driving system are expressed as anautonomous driving system 1000.

1.3. Elements of Autonomous Driving System (AD/ADAS)

The autonomous driving system 1000 may be mounted inside a vehicle 100.Also, the autonomous driving system 1000 may be mounted inside anaircraft, a ship, or an unmanned aerial vehicle as well as the vehicle100, but the present invention is not limited thereto.

FIG. 1 is a diagram illustrating elements of an autonomous drivingsystem according to an embodiment.

Referring to FIG. 1 , an autonomous driving system 1000 according to anembodiment may include various elements.

For example, the autonomous driving system 1000 may include at least onecontroller 1100, at least one communication module 1200, at least onesensor 1300, at least one infotainment system 1400, etc., but thepresent invention is not limited thereto.

Hereinafter, various examples of the elements of the autonomous drivingsystem 1000 will be described in detail.

1.3.1. Controller

Referring to FIG. 1 again, the autonomous driving system 1000 accordingto an embodiment may include at least one controller 1100.

Also, the controller 1100 may control elements of an apparatus includingthe controller 1100. For example, the controller 1100 may control atleast one sensor 1300 or at least one communication module 1200 includedin the autonomous driving system 1000, but the present invention is notlimited thereto.

Also, the controller 1100 may acquire data from the at least one sensor1300 or the at least one communication module 1200. For example, thecontroller 1100 may acquire data from a light detection and ranging(LiDAR) device located in a vehicle, but the present invention is notlimited thereto. The controller may acquire data from various sensorsand a communication module.

Also, the controller 1100 may be used to control a vehicle. For example,the controller 1100 may control the speed, direction, path, or the likeof the vehicle, but the present invention is not limited thereto. Thecontroller 1100 may control the various operations of the vehicle.

Also, the controller 1100 may be expressed as an ECU, a processor, orthe like depending on the embodiment, but the present invention is notlimited thereto.

Also, in this specification, the controller 1100 may refer to acontroller of a device where the autonomous driving system 1000 isplaced and may also refer to a controller placed in at least one sensor.However, the present invention is not limited thereto, and thecontroller 1100 may collectively refer to at least one controller placedin the autonomous driving system 1000.

1.3.2. Communication Module

Referring to FIG. 1 again, the autonomous driving system 1000 accordingto an embodiment may include at least one communication module 1200.

In this case, the at least one communication module 1200 may be used toshare at least one piece of data with other devices. As an example, thecontroller 1100 may transmit or receive data to or from the outsidethrough the at least one communication module 1200.

Also, the at least one communication module 1200 may be used toimplement at least one vehicle-to-everything (V2X) system. In detail,the communication module 1200 may be used to implement at least one V2Xsystem such as a vehicle-to-vehicle (V2V) system, a vehicle-to-infra(V2I) system, a vehicle-to-network (V2N) system, a vehicle-to-pedestrian(V2P) system, and a vehicle-to-cloud (V2C) system.

Also, the autonomous driving system 1000 may share data acquired fromthe at least one sensor 1300 and relevant property data through the atleast one communication module 1200, but the present invention is notlimited thereto.

Also, the at least one communication module 1200 may include at leastone antenna. For example, the at least one communication module mayinclude at least one of Global Positioning System (GPS), GlobalNavigation Satellite System (GNSS), Amplitude Modulation (AM), FrequencyModulation (FM), Fourth Generation (4G), and Fifth Generation (5G)antennas, but the present invention is not limited thereto.

1.3.3. Sensor

Referring to FIG. 1 again, the autonomous driving system 1000 accordingto an embodiment may include at least one sensor 1300.

Also, the at least one sensor 1300 according to an embodiment may beused to acquire vehicle surrounding information.

For example, the at least one sensor may be used to acquire distanceinformation of an object near a vehicle, but the present invention isnot limited thereto. The sensor may be used to acquire various pieces ofinformation about an object near a vehicle.

FIG. 2 is a diagram specifically illustrating at least one sensoraccording to an embodiment.

Referring to FIG. 2 , the at least one sensor 1300 may include at leastone LiDAR device 1310, at least one camera device 1320, at least oneradar device 1330, at least one ultrasonic sensor 1340, at least one GPSsensor 1350, at least one inertial measurement unit 1360, and the like.It will be appreciated that the type of the sensor is not limitedthereto, and the at least one sensor 1300 may include all or only someof the above-described sensors 1310, 1320, 1330, 1340, 1350, and 1360.

Referring to FIG. 2 again, the at least one sensor 1300 may include atleast one LiDAR device 1310.

In this case, the LiDAR device 1310 may be defined as a device thatmeasures a distance to an object using laser beams.

More specifically, the at least one LiDAR device 1310 may output a laserbeam. When the output laser beam is reflected by an object, the LiDARdevice 1310 may receive the reflected laser beam and measure thedistance between the object and the LiDAR device 1310. Here, the LiDARdevice 1310 may measure the distance to the object by using variousschemes such as a triangulation scheme and a Time-of-Flight (TOF)scheme.

Also, the LiDAR device 1310 may include a laser beam output unit.

In this case, the laser beam output unit may emit a laser beam. Also,the laser beam output unit may include one or more laser beam outputelements. Also, the laser beam output units may include a laser diode(LD), a solid-state laser, a high power laser, a light-emitting diode(LED), a vertical-cavity surface-emitting laser (VCSEL), an externalcavity diode laser (ECDL), etc., but the present invention is notlimited thereto.

Also, the LiDAR device 1310 may include a light-receiving unit.

In this case, the light-receiving unit may detect a laser beam. Forexample, the light-receiving unit may detect a laser beam reflected byan object located in a scanning region. Also, the light-receiving unitmay receive a laser beam and generate an electric signal on the basis ofthe received laser beam. For example, the sensor 1300 may include a PNphotodiode, a phototransistor, a PIN photodiode, an avalanche photodiode(APD), a single-photon avalanche diode (SPAD), silicon photomultipliers(SiPM), a comparator, a complementary metal-oxide-semiconductor (CMOS),a charge-coupled device (CCD), or the like, but the present invention isnot limited thereto.

Also, the LiDAR device 1310 may include an optical system.

In this case, the optical system may change a flight path of a laserbeam. For example, the optical system may change a flight path of alaser beam emitted from the laser beam output unit such that the laserbeam is directed to a scanning region.

Also, the optical system may change a flight path of a laser beam byreflecting the laser beam. In this case, the optical system may includea first scanner for performing a scan in a first direction and a secondscanner for performing a scan in a second direction. Also, the opticalsystem may include a rotational optic for performing a scan whilerotating both of the laser beam output unit and the light-receivingunit.

For example, the optic system may include a mirror, a resonance scanner,a micro-electromechanical system (MEMS) mirror, a voice coil motor(VCM), a polygonal mirror, a rotating mirror, a Galvano mirror, or thelike, but the present invention is not limited thereto.

Also, the optical system may change a flight path of a laser beam byrefracting the laser beam. For example, the optical system may includelenses, prisms, microlenses, microfluidic lenses, or the like, but thepresent invention is not limited thereto.

Also, the optical system may change a flight path of a laser beam bychanging the phase of the laser beam. For example, the optical systemmay include an optical phased array (OPA), a metalens, a metasurface, orthe like, but the present invention is not limited thereto.

Also, the at least one LiDAR device 1310 may be placed in variouspositions of a vehicle so as to secure a field of view of thesurroundings of the vehicle. For example, the LiDAR device 1310 mayinclude a plurality of LiDARs 1311 to 1314. The plurality of LiDARs 1311to 1314 may include one or multiple LiDARs placed in each of variouspositions, e.g., the front, the rear, the side, and the roof of thevehicle.

In detail, when the first LiDAR 1311 is placed on the front of thevehicle, the first LiDAR 1311 may detect distance information regardingan object located in front of the vehicle, and the first LiDAR 1311 maybe placed on a headlamp, a front bumper, a grille, or the like of thevehicle, but the present invention is not limited thereto.

Also, when the second LiDAR 1312 is placed on the side of the vehicle,the second LiDAR 1312 may detect distance information of an objectlocated to the side of the vehicle, and the second LiDAR 1312 may beplaced on a side mirror, a side garnish, or the like of the vehicle, butthe present invention is not limited thereto.

Also, when the third LiDAR 1313 is placed on the rear of the vehicle,the third LiDAR 1313 may detect distance information of an objectlocated behind the vehicle, and the third LiDAR 1313 may be placed on arear bumper, a brake light, or the like of the vehicle, but the presentinvention is not limited thereto.

Also, when the fourth LiDAR 1314 is placed on the roof of the vehicle,the fourth LiDAR 1314 may detect distance information of an objectlocated in front of, behind, and to the side of the vehicle, and thefourth LiDAR 1314 may be placed on a sunroof, roof, or the like of thevehicle, but the present invention is not limited thereto.

Referring to FIG. 2 again, the at least one sensor 1300 according to anembodiment may include at least one camera device 1320.

In this case, the at least one camera device 1320 may acquire shapeand/or color information regarding an object located near a vehicleequipped with the autonomous driving system 1000.

Also, the at least one camera device 1320 may be placed in variouspositions of a vehicle so as to secure shape and/or color informationregarding the surroundings of the vehicle and the interior of thevehicle. For example, the camera device 1320 may include a plurality ofcameras 1321 to 1323. The plurality of cameras 1321 to 1323 may includeone or multiple cameras placed in each of various positions, e.g., thefront, the side, the rear, and the inside of the vehicle.

In detail, when the first camera 1321 is placed on the front of thevehicle, the first camera 1321 may detect shape and/or color informationregarding an environment in front of the vehicle, and the first camera1321 may be placed on a black box, a headlight, a grille, or the like ofthe vehicle, but the present invention is not limited thereto.

Also, when the second camera 1322 is placed on the rear of the vehicle,the second camera 1322 may detect shape and/or color informationregarding an environment behind the vehicle, and the second camera 1322may be placed on a rear bumper, a brake light, or the like of thevehicle, but the present invention is not limited thereto.

Also, when the third camera 1323 is placed inside the vehicle, the thirdcamera 1323 may detect shape and/or color information regarding anenvironment inside the vehicle, and the third camera 1323 may be placedon a black box, a room mirror, or the like of the vehicle, but thepresent invention is not limited thereto.

Also, the camera device 1320 may include a stereo camera. Here, thestereo camera may refer to a camera for determining a distance to anobject as well as the shape of the object using a plurality of cameras.

Also, the camera device 1320 may include a time-of-flight (ToF) camera.Here, a ToF camera may refer to a camera capable of determining adistance to an object by employing time-of-flight techniques.

Referring to FIG. 2 again, the at least one sensor 1300 according to anembodiment may include at least one radar device 1330.

In this case, the at least one radar device 1330 may be a device fordetecting a distance to an object and a position of an object usingelectromagnetic waves.

Also, the at least one radar device 1330 may include various types ofradar devices in order to acquire accurate distance information ofobjects located at long distances from the vehicle, objects located atmedium distances, and objects located at short distances. For example,the at least one radar device 1330 may include a first radar 1331 foracquiring distance information of objects located at long distances, asecond radar 1332 for acquiring distance information of objects locatedat medium or short distances, etc., but the present invention is notlimited thereto.

Also, the at least one radar device 1330 may be placed in variouspositions of a vehicle so as to secure a field of view of thesurroundings of the vehicle. For example, the at least one radar device1330 may be placed on the front, the rear, or the side of the vehicle,but the present invention is not limited thereto.

Referring to FIG. 2 again, the at least one sensor 1300 according to anembodiment may include at least one ultrasonic sensor 1340.

In this case, the at least one ultrasonic sensor 1340 may be a devicefor detecting whether an object is present near a vehicle.

Also, the at least one ultrasonic sensor 1340 may be placed in variouspositions of a vehicle so as to detect whether an object is present nearthe vehicle. For example, the at least one ultrasonic sensor 1340 may beplaced on the front, the rear, or the side of the vehicle, but thepresent invention is not limited thereto.

Referring to FIG. 2 again, the at least one sensor 1300 according to anembodiment may include at least one GPS sensor 1350.

In this case, the at least one GPS sensor 1350 may be a device forfinding the global position of a vehicle. In detail, the at least oneGPS sensor 1350 may forward global position information of the GPSsensor 1350 to the controller 1100 through the Global PositioningSystem.

Referring to FIG. 2 again, the at least one sensor 1300 according to anembodiment may include at least one inertial measurement unit (IMU)1360.

In this case, the at least one IMU 1360 is an electronic device thatmeasures and reports a specific force and an angular ratio of a vehicleand a magnetic field surrounding a vehicle by using a combination of anaccelerometer, a tachometer, and a magnetometer. In detail, the at leastone IMU 1360 may be activated by detecting a linear acceleration usingat least one accelerometer and by detecting a rotational speed using atleast one gyroscope.

1.3.4. Infotainment System

Referring to FIG. 1 again, the autonomous driving system 1000 accordingto an embodiment may include at least one infotainment system 1400.

In this case, the at least one infotainment system 1400 according to anembodiment may display at least one piece of information to an occupant.

FIG. 3 is a diagram showing a display scheme through an infotainmentsystem according to an embodiment.

Referring to FIG. 3 , the infotainment system 1400 according to anembodiment may include a high-definition map 1420, a message window1430, a screen 1410 for showing the high-definition map 1420 and themessage window 1430 to an occupant, an information field 1440 forproviding object information acquired from a sensor, etc., but thepresent invention is not limited thereto.

Referring to FIG. 3 again, the infotainment system 1400 according to anembodiment may include a high-definition map that shows positioninformation of a host vehicle and position information of a nearbyobject.

In this case, the high-definition map 1420 may be downloaded by thecontroller 1100. In detail, the high-definition map 1420 may begenerated by and stored in an external server, and the controller 1100may download the high-definition map 1420 and display thehigh-definition map 1420 to an occupant through the infotainment system1400.

Also, the high-definition map 1420 may be generated based on sensor dataacquired from the at least one sensor 1300 included in the autonomousdriving system 1000. In detail, the LiDAR device 1310 included in the atleast one sensor 1300 may acquire distance information of an objectoutside the vehicle. In this case, the controller 1100 may generate ahigh-definition map 1420 including the position information of theobject outside the vehicle on the basis of the distance information andmay display the high-definition map 1420 to an occupant through theinfotainment system 1400.

Also, the controller 1100 may generate the high-definition map using thesensor data on the basis of a downloaded map. In detail, the controller1100 may implement the high-definition map 1420 by generating positioninformation of the object using the sensor data and by showing theposition information of the object in the downloaded map and then maydisplay the high-definition map 1420 to an occupant through theinfotainment system 1400.

Referring to FIG. 3 again, the infotainment system 1400 according to anembodiment may include a message window 1430 for displaying, to a user,a message transmitted from the outside.

For example, the message window 1430 may include a message received fromthe outside, information to be forwarded to an occupant, an interfacefor receiving an input from an occupant, information indicating whetherdata transmission is approved by an external server, etc., but thepresent invention is not limited thereto.

More specifically, when a request message for sensor data is receivedfrom an external server, the controller 1100 may display the requestmessage through the message window 1430. In this case, an occupant mayenter an input for transmitting the sensor data in response to therequest message.

Also, when a notification message indicating that a traffic event hasoccurred is received from an external server, the controller 1100 maydisplay the notification message through the message window 1430.

Also, the message window 1430 may be displayed on a separate screendifferent from that of the high-definition map 1420.

Also, the message window 1430 may be displayed on the same screen as thehigh-definition map 1420. In detail, the message window 1430 may bedisplayed so as not to overlap the high-definition map 1420, but thepresent invention is not limited thereto.

Referring to FIG. 3 again, the infotainment system 1400 according to anembodiment may include a screen 1410 for showing the high-definition map1420 and the message window 1430.

Also, the screen 1410 may include a touch sensor, an input button, etc.,but the present invention is not limited thereto.

In this case, when a touch input is received from an occupant, thescreen 1410 may transmit the content of the touch input of the occupantto the controller 1100. For example, when the controller 1100 forwards,to the occupant through the message window 1430, a request message forsensor data received from an external server, the occupant may enter aresponse to the request message by touching the screen 1410.

Also, when the controller 1100 displays, through the message window1430, a notification message for a traffic event received from anexternal server, the occupant may enter an input indicating whether thenotification message is confirmed.

Referring to FIG. 3 again, the infotainment system 1400 according to anembodiment may include an information field 1440 for showing informationacquired from the at least one sensor 1300 in the windshield of avehicle. In this case, the windshield may include an electronic screento show the information field 1440.

More specifically, in order to forward information acquired through theat least one sensor 1300 to an occupant, the controller 1100 may showthe information field 1440 in the windshield of the vehicle through theinfotainment system 1400.

Also, the information field 1440 may show class information, speed,movement direction, etc. that are acquired when a LiDAR device includedin the at least one sensor 1300 scans an object, but the presentinvention is not limited thereto. The information field 1440 may furtherinclude a plurality of pieces of information acquired by varioussensors.

Also, the information field 1440 may be displayed on the screen 1410 orthe windshield in an augmented reality (AR) scheme or a virtual reality(VR) scheme.

1.4. Autonomous Driving System 1.4.1. Autonomous Driving System UsingSensor 1.4.1.1. Overview

An autonomous driving system 1000 may drive a vehicle with no or minimumdriver intervention on the basis of sensor data acquired using at leastone sensor 1300. For example, the autonomous driving system 1000 mayautonomously drive a vehicle on the basis of data acquired using atleast one of at least one LiDAR device 1310, at least one camera device1320, at least one radar device 1330, and at least one ultrasonic sensor1340 which are placed inside the vehicle.

Also, the autonomous driving system 1000 may perform simultaneouslocalization and mapping (SLAM)-based autonomous driving andhigh-definition-map-based autonomous driving on the basis of the sensordata, but the present invention is not limited thereto.

In detail, a vehicle that performs the SLAM-based autonomous driving maytravel autonomously by recognizing a surrounding environment through theat least one sensor 1300, creating a map of a corresponding space, andaccurately determining its own position.

In addition, a vehicle that performs high-definition-map-basedautonomous driving may travel autonomously by recognizing an object nearthe vehicle on the basis of a high-precision map acquired from thecontroller 1100.

Also, the autonomous driving system 1000 may perform pedestriandetection, collision avoidance, traffic information recognition, parkingassistance, surround view, proximity collision risk detection, etc.through the at least one sensor 1300, but the present invention is notlimited thereto.

Hereinafter, specific examples of the autonomous driving system using atleast one sensor will be described in detail.

1.4.1.2. Autonomous Driving System for Safety

The autonomous driving system 1000 may include a system for the safetyof pedestrians and occupants of a vehicle equipped with the autonomousdriving system 1000. Also, the safety system may operate based on sensordata acquired from at least one sensor 1300 included in the autonomousdriving system 1000.

The description of the autonomous driving system for safety is aboutvarious examples controlled by an autonomous vehicle and may beimplemented with the following descriptions in Sections 2 to 5.

The autonomous driving system 1000 may detect a driving pattern of anearby moving object and then detect an abnormal driving pattern of themoving object.

FIG. 4 is a diagram showing a situation in which an autonomous drivingsystem detects a moving object showing an abnormal driving patternaccording to an embodiment.

Referring to FIG. 4 , a first vehicle 101 equipped with the autonomousdriving system 1000 may detect a driving pattern of a nearby objectthrough at least one sensor 1300 included in the autonomous drivingsystem 1000.

More specifically, the controller 1100 included in the autonomousdriving system 1000 may detect a driving pattern of a second vehicle 102located near the first vehicle 101 on the basis of sensor data acquiredfrom the at least one sensor 1300.

Also, the controller 1100 may track the movement of the second vehicle102 in order to detect an abnormal driving pattern of the second vehicle102. In detail, when the speed and direction of the second vehicle 102irregularly change, the controller 1100 may control the at least onesensor 1300 to track the movement of the second vehicle 102.

Also, the controller 1100 may determine whether the driving pattern ofthe second vehicle 102 is abnormal on the basis of the sensor data. Indetail, the controller 1100 may acquire movement information includingthe speed and direction of the second vehicle 102 through the at leastone sensor 1300. In this case, the controller 1100 may determine thatthe change in the speed and direction of the second vehicle 102 isabnormal on the basis of the movement information.

Also, the controller 1100 may set a driving-related threshold to detectan abnormal driving pattern of the second vehicle 102. In detail, thecontroller 1100 may quantify the movement of the second vehicle 102acquired through the at least one sensor 1300 and compare the quantifiedmovement to the threshold. In this case, when the movement of the secondvehicle 102 exceeds the threshold, the controller 1100 may determinethat the second vehicle 102 has an abnormal driving pattern.

Also, when the abnormal driving pattern of the second vehicle 102 isdetected, the controller 1100 may control the first vehicle 101 to avoida collision with the second vehicle 102. For example, the controller1100 may decelerate the first vehicle 101, accelerate the first vehicle101, or re-route the first vehicle 101, but the present invention is notlimited thereto.

Also, when the at least one sensor 1300 is the LiDAR device 1310, thecontroller 1100 may detect a moving object having an abnormal drivingpattern by utilizing distance information acquired through the LiDARdevice 1310. In this case, the controller 1100 may generate informationregarding the position and speed of an object present in the field ofview of the LiDAR device 1310 on the basis of distance information ofthe object.

In detail, the autonomous driving system 1000 may generate a vector mapof a nearby object using data acquired from the LiDAR device 1310. Inmore detail, the controller 1100 may acquire a vector map including thespeed and the like of the second vehicle 102 on the basis of distanceinformation of the second vehicle 102 acquired by the LiDAR device 1310.

Also, the autonomous driving system 1000 may determine whether thesecond vehicle 102 has an abnormal driving pattern using the vector map.Also, the controller 1100 may control the first vehicle on the basis ofthe vector map.

Also, the autonomous driving system 1000 may compute a space where avehicle can move in case an emergency occurs in the vicinity.

FIG. 5 is a diagram showing a situation in which an autonomous drivingsystem recognizes an accident of a vehicle in front while drivingaccording to an embodiment.

Referring to FIG. 5 , a first vehicle 103 equipped with the autonomousdriving system 1000 may detect a space where the first vehicle 103 canmove through at least one sensor 1300 included in the autonomous drivingsystem 1000.

In detail, a controller 1100 included in the autonomous driving system1000 may pre-compute a space 200 where the first vehicle 103 can move onthe basis of sensor data acquired from the outside or the at least onesensor 1300. In detail, the controller 1100 may compute spaces where noobject is detected and which has a predetermined volume in a spaceindicated by the sensor data.

Also, the controller 1100 may select a space in which the first vehicle103 can travel from among the computed spaces and store the selectedspace. For example, when the available space 200 is in a diagonaldirection of the first vehicle 103, the controller 1100 may storeinformation related to the available space 200. However, the presentinvention is not limited thereto, and the controller 1100 may storeinformation related to the space 200 where the first vehicle 103 canmove without risk of collision with a nearby object among spaces whichare not set as the driving path of the first vehicle 103.

Also, when an emergency accident occurs in front of the first vehicle103, the controller 1100 may move the first vehicle 103 to the availablespace 200 using previously stored space information.

Also, when the controller 1100 recognizes the occurrence of theemergency near the first vehicle 103, the controller 1100 may computethe space 200 where the first vehicle 103 can move.

In detail, when the controller recognizes an accident between a secondvehicle 104 and a third vehicle 105 on the basis of the sensor data, thecontroller 1100 may compute the space 200 where the first vehicle 103can move. In this case, the controller 1100 may recognize the accidentthrough a relative position between a set of data corresponding to thesecond vehicle 104 and a set of data corresponding to the third vehicle105, which are included in the sensor data, but the present invention isnot limited thereto.

Also, when the controller 1100 computes the space 200 where the firstvehicle 103 can move, the controller 1100 may control the first vehicle103 to move the first vehicle 103 to the available space 200. Forexample, the controller 1100 may control the steering of the firstvehicle 103 to move the first vehicle 103 to the available space 200,but the present invention is not limited thereto.

Also, when the at least one sensor 1300 is the LiDAR device 1310, thecontroller 1100 may acquire empty-space data using data acquired fromthe LiDAR device 1310. In this case, the controller 1100 may generateinformation regarding the position and speed of an object placed in thefield of view of the LiDAR device 1310 on the basis of distanceinformation of the object.

In detail, the controller 1100 may generate a three-dimensional (3D) mapusing position information of an object near the first vehicle 103 whichis acquired by the LiDAR device 1310. In this case, the controller 1100may store a space of the 3D map where there is no object data as dataregarding the available space 200. Also, when an emergency occurs nearthe first vehicle 103, the controller 1100 may move the first vehicle103 to the available space 200 using the stored space data.

Also, when the autonomous driving system 1000 recognizes that a secondvehicle 107 located in front of a first vehicle 106 is suddenly movingbackward, the autonomous driving system 1000 may control the firstvehicle 106 to avoid a collision with the second vehicle 107.

FIG. 6 is a diagram showing a situation in which an autonomous drivingsystem recognizes a sudden backward movement of a vehicle in frontaccording to an embodiment.

Referring to FIG. 6 , the first vehicle 106 equipped with the autonomousdriving system 1000 may detect the movement of the second vehicle 107through the at least one sensor 1300 included in the autonomous drivingsystem 1000.

For example, the controller 1100 included in the autonomous drivingsystem 1000 may detect a movement direction of the second vehicle 107located in front of the first vehicle 106 on the basis of the sensordata acquired from the at least one sensor 1300.

More specifically, the controller 1100 may acquire movement informationincluding the movement speed and movement direction of the secondvehicle 107 through the at least one sensor 1300. In this case, when thecontroller 1100 determines that the second vehicle 107 moves backward onthe basis of the movement information, the controller 1100 may transmita notification for warning the second vehicle 107 to the second vehicle107.

Also, the controller 1100 may sound a horn to warn the second vehicle107.

Also, when the controller 1100 determines that there is a space wherethe first vehicle 106 can move behind the first vehicle 106, thecontroller 1100 may move the first vehicle 106 to the space to whichmovement is possible.

Also, when the at least one sensor 1300 is the LiDAR device 1310, thecontroller 1100 may detect whether the second vehicle 107 moves backwardusing data acquired from the LiDAR device 1310. In this case, thecontroller 1100 may generate movement information indicating themovement direction and movement speed of the second vehicle 107 on thebasis of position information of the second vehicle 107 located in thefield of view of the LiDAR device 1310.

More specifically, the controller 1100 may determine whether the secondvehicle 107 moves backward on the basis of the movement information ofthe second vehicle 107. For example, when the second vehicle 107approaches the first vehicle 106, the controller 1100 may determine thatthe second vehicle 107 is moving backward. Also, when the distancebetween the first vehicle 106, which is stopped, and the second vehicle107 decreases, the controller 1100 may determine that the second vehicle170 is moving backward.

Also, the autonomous driving system 1000 may detect a change in thedirection of a second vehicle 109 located near a first vehicle 108.

FIG. 7 is a diagram showing a situation in which an autonomous drivingsystem tracks the movement of a vehicle's wheel according to anembodiment.

Referring to FIG. 7 , the first vehicle 108 equipped with the autonomousdriving system 1000 may detect a change in the direction of the secondvehicle 109 through at least one sensor 1300 included in the autonomousdriving system 1000.

For example, the controller 1100 included in the autonomous drivingsystem 1000 may detect a change in the direction of the second vehicle109 by detecting a wheel 109 a of the second vehicle 109 located nearthe first vehicle 108 using sensor data acquired through the at leastone sensor 1300

In this case, when an object included in the sensor data is determinedas the wheel 109 a of the second vehicle, the controller 1100 may trackthe wheel 109 a of the second vehicle. Also, the controller 1100 maycontrol a scan pattern of the at least one sensor 1300 to continuouslyacquire sensor data regarding the wheel 109 a of the second vehicle.

Also, when the wheel 109 a of the second vehicle is directed to thefront of the first vehicle 108, the controller 1100 may control thefirst vehicle 108 to prevent the first vehicle 108 from colliding withthe second vehicle 109. For example, the controller 1100 may deceleratethe first vehicle 108 or re-route the first vehicle 108, but the presentinvention is not limited thereto.

Also, when the direction of the wheel 109 a of the second vehiclechanges suddenly, the controller 1100 may decelerate the first vehicle108 or re-route the first vehicle 108 regardless of the currentdirection of the wheel 109 a of the second vehicle.

Also, when the at least one sensor 1300 is the LiDAR device 1310, thecontroller 1100 may detect a change in the direction of the wheel 109 aof the second device using data acquired from the LiDAR device 1310. Inthis case, the controller 1100 may detect the movement of the wheel 109a of the second vehicle using temporal position information of the wheel109 a of the second wheel located in the field of view of the LiDARdevice 1310.

More specifically, the controller 1100 may generate a 3D map includingdata on the wheel 109 a of the second vehicle or predicted movementinformation of the second vehicle 109 which is predicted through thedata on the wheel 109 a of the second vehicle by using the LiDAR device1310. In this case, the 3D map may include position information of thewheel 109 a of the second vehicle that changes over time. Also, thecontroller 1100 may detect the change in the direction of the wheel 109a of the second vehicle using the 3D map to control the first vehicle108.

Also, the autonomous driving system 1000 may detect a risk factor of aroad on which a first vehicle 110 is traveling (e.g., a crack in theroad or black ice present on the road).

FIG. 8 is a diagram illustrating a method of detecting, by an autonomousdriving system, black ice present on a road according to an embodiment.

Referring to FIG. 8 , the first vehicle 110 equipped with the autonomousdriving system 1000 may detect a surface condition of the road on whichthe first vehicle 110 is traveling through the at least one sensor 1300included in the autonomous driving system 1000.

For example, the controller 1100 included in the autonomous drivingsystem 1000 may detect a crack in the road on which the first vehicle istraveling on the basis of the sensor data acquired from the at least onesensor 1300. Also, the controller 1100 may detect black ice present onthe road on the basis of the sensor data, but the present invention isnot limited thereto.

Also, the LiDAR device 1310 included in the at least one sensor 1300 mayacquire sensor data including intensity information associated with thereflectance of an object. In detail, the sensor data may includeintensity information of a first region 300 included in the field ofview of the at least one sensor 1300. In this case, the intensityinformation may include an intensity value 311, which is a valuecorresponding to the reflectance of the object. Also, a mean, adeviation, and a standard deviation may be used as the intensity valueincluded in the intensity information, and at least one piece of datamay be amplified, but the present invention is not limited thereto.

Also, the controller 1100 may determine the risk of the road on thebasis of an intensity distribution chart 310 representing aspace-specific intensity distribution chart 310 of intensity valuesincluded in the intensity information. In this case, the intensitydistribution chart 310 may include an intensity value 311 for each pointof the first region 300.

Also, when the intensity value 311 changes rapidly with respect to apredetermined boundary 312 in the intensity distribution chart 310 ofthe first region 300, the controller 1100 may determine that a regionwithin the predetermined boundary 312 is a dangerous region.

Also, the controller 1100 may set an intensity threshold using theaverage of intensity values for each region of the road. In detail, thecontroller 1100 may compute the average of intensity values of eachpoint on the road on which the vehicle is traveling and may set anintensity threshold on the basis of the average. In this case, thecontroller 1100 may compare the intensity threshold to the average ofthe intensity values of each point of the first region 300. Also, whenthe comparison result is that the average of the intensity values of thefirst region 300 is greater than or equal to the intensity threshold,the controller 1100 may determine that the first region 300 is adangerous region.

Also, the controller 1100 may adjust sensor activation energy in orderto detect a road risk using the sensor data acquired through the atleast one sensor 1300. For example, the controller 1100 may adjust thesensor activation energy, detect a corresponding pattern, and detect aroad risk, but the present invention is not limited thereto.

Also, the dangerous region may include a region that may becomedangerous to the driving of the first vehicle 110. For example, thedangerous region may include a region having black ice and a regionhaving a road crack, but the present invention is not limited thereto.

Also, the autonomous driving system 1000 may detect an illegally parkedvehicle through sensor data.

More specifically, when a vehicle is stopped on a road, the autonomousdriving system 1000 may determine whether a space associated with thestopped vehicle is an available parking space and may determine that thevehicle is an illegally parked vehicle when the vehicle is stopped for apredetermined time or more even though the space is not an availableparking space.

In this case, the autonomous driving system 1000 may detect a parkingline through at least one sensor 1300 and determine whether parking isavailable on the basis of the detected parking line. Also, theautonomous driving system 1000 may determine an available parking regionusing a prestored map.

Also, the autonomous driving system 1000 may compare the width of a roadto the width of a first vehicle 111 equipped with the autonomous drivingsystem 1000 and may determine whether the first vehicle 111 can travelon the road.

FIG. 9 is a diagram showing a situation in which a vehicle equipped withan autonomous driving system detects an illegally parked vehicleaccording to an embodiment.

Referring to FIG. 9 , the controller 1100 included in the autonomousdriving system 1000 may determine whether the first vehicle 111 can movewhile avoiding a second vehicle 112 which is illegally parked on a road.

In detail, the controller 1100 included in the autonomous driving system1000 may compute a space in which the first vehicle 111 can travel onthe basis of sensor data acquired from the at least one sensor 1300. Forexample, when the second vehicle 112 is stopped on a road on which thefirst vehicle 111 is traveling, the controller 1100 may compare thewidth pa of the travelable road to the width pb of the first vehicle. Inthis case, the width pb of the first vehicle may be prestored in thecontroller 1100.

Also, when the width pa of the road is greater than the width pb of thefirst vehicle by a predetermined length or more, the controller 1100 maycontrol the first vehicle 111 such that the first vehicle 111 can travelon the traveling road while avoiding the second vehicle 112.

Also, the controller 1100 may determine a space between a center lineand the second vehicle 112 on the basis of the sensor data. In thiscase, the controller 1100 may determine whether the space is a spacethrough which the first vehicle 111 can pass and then may control thefirst vehicle.

Also, when the at least one sensor 1300 is the LiDAR device 1310, thecontroller 1100 may detect a space in which the first vehicle 111 cantravel on the basis of distance information acquired from the LiDARdevice 1310. In this case, the controller 1100 may generate positioninformation of the center line and the second vehicle 112 on the basisof distance information of the centerline and the second vehicle 112.

More specifically, the controller 1100 may generate a 3D map on thebasis of the sensor data acquired from the LiDAR device 1310. In thiscase, the controller 1100 may determine a space in which the firstvehicle 111 can travel on the basis of the 3D map.

Also, the autonomous driving system 1000 may detect an objectapproaching a vehicle equipped with the autonomous driving system 1000within a dangerous radius. In detail, the autonomous driving system 1000may determine the speed, direction, and the like of a two-wheeledvehicle approaching in the vicinity on the basis of the sensor dataacquired from the at least one sensor 1300. In this case, the controller1100 may display the speed and direction of the two-wheeled vehicle toan occupant through the infotainment system.

Also, when the controller 1100 determines that the two-wheeled vehicleis located within a dangerous radius on the basis of the speed anddirection, the controller may inform an occupant of the presence of thetwo-wheeled vehicle. For example, the controller 1100 may perform anoperation of locking the doors of the vehicle, an operation of notifyingof danger through the infotainment system 1400, an operation ofdisplaying the presence of the two-wheeled vehicle to the side mirror ofthe vehicle, and the like, but the present invention is not limitedthereto.

Also, the autonomous driving system 1000 may further include ashort-range LiDAR device in order to clearly determine the presence ofthe two-wheeled vehicle. In this case, the short-range LiDAR device mayacquire distance information of an object close to the vehicle andprovide the distance information to the controller 1100. However, thepresent invention is not limited thereto, and the autonomous drivingsystem 1000 may further include at least one sensor for detecting anearby object.

Also, a first vehicle equipped with the autonomous driving system 1000may detect a situation in which an oncoming vehicle makes a suddenU-turn through the sensor data. In detail, the controller 1100 includedin the autonomous driving system 1000 may form a vector map includingthe speed and direction of a second vehicle, which is oncoming, throughsensor data acquired from a LiDAR device included in the at least onesensor 1300. Also, the autonomous driving system 1000 may detect whetherthe second vehicle is making a U-turn using the vector map. Also, whenthe second vehicle makes a sudden U-turn, the controller 1100 maycontrol the speed of the first vehicle.

Also, before the first vehicle equipped with the autonomous drivingsystem 1000 departs, the autonomous driving system 1000 may detectwhether there is an object near the first vehicle.

More specifically, the controller 1100 included in the autonomousdriving system 1000 may control at least one sensor 1300 to determinewhether there is an object near the first vehicle before moving thefirst vehicle. For example, when a cat is present under the firstvehicle, the at least one sensor 1300 may detect the presence of the catand transmit the presence of the cat to the controller 1100. In thiscase, the controller 1100 may stop the first vehicle until the catleaves.

Also, the autonomous driving system 1000 may track a pedestrian near thefirst vehicle equipped with the autonomous driving system 1000 andprepare for a dangerous situation. Here, the pedestrian may includevarious people such as men, women, children, and the elderly. Accordingto an embodiment, the autonomous driving system 1000 may identify thetype of the pedestrian.

In detail, the controller 1100 included in the autonomous driving system1000 may detect the movement of a pedestrian within a predetermineddistance from the vehicle through at least one sensor 1300. Also, whenthe pedestrian disappears from the field of view of the at least onesensor, the controller 1100 may generate tracking data for predictingthe movement direction of the pedestrian by using already acquiredposition information of the pedestrian.

Also, the controller 1100 may prestore a control method to prepare for asituation in which the pedestrian suddenly enters a road on the basis ofthe tracking data. For example, the control method may include stoppingthe vehicle or changing a path of the vehicle, but the present inventionis not limited thereto.

Also, the autonomous driving system 1000 may determine a region relatedto legal regulations such as a child protection zone and control thevehicle. In detail, the autonomous driving system 1000 may determine achild protection zone by scanning a sign indicating the child protectionzone through at least one sensor 1300. Also, the autonomous drivingsystem 1000 may determine a child protection zone using prestoredinformation related to the child protection zone. In this case, when thevehicle equipped with the autonomous driving system 1000 enters a childprotection zone, the controller 1100 may control the vehicle to travelat a predetermined speed or less.

1.4.1.3. Autonomous Driving System for Convenience

The autonomous driving system 1000 may include a system for theconvenience of occupants of a vehicle equipped with the autonomousdriving system 1000. Also, the system for the convenience may operatebased on sensor data acquired from at least one sensor 1300 included inthe autonomous driving system 1000.

The description of the autonomous driving system for the convenience isabout various examples controlled by an autonomous vehicle and may beimplemented with the following descriptions in Sections 2 to 6.

The autonomous driving system 1000 may detect an available parking spaceto assist an occupant in parking the vehicle.

FIG. 10 is a diagram showing a situation in which an autonomous drivingsystem detects an available parking space according to an embodiment.

Referring to FIG. 10 , a first vehicle 113 equipped with the autonomousdriving system 1000 may detect an available parking space through the atleast one sensor 1300.

Also, the controller 1100 included in the autonomous driving system 1000may detect a parking line 10 on the basis of sensor data acquired fromthe at least one sensor 1300. For example, the controller 1100 mayacquire intensity information associated with the reflectance of anobject through the LiDAR device 1310 included in the at least one sensor1300. In this case, the controller 1100 may determine that the object isthe parking line 10 on the basis of the intensity information.

Also, the controller 1100 may detect whether an obstacle is present in aspace formed in the detected parking line 10. In this case, when noobstacle is present in the space formed in the parking line 10, thecontroller 1100 may determine that the space is an available parkingspace.

Also, the controller 1100 may detect an available parking space bydetecting a second vehicle 114, which has been parked, on the basis ofthe sensor data. In detail, when data 20 corresponding to an exterior ofthe parked second vehicle is included in the sensor data, the controller1100 may not determine whether the second vehicle 114 is present in anavailable parking position.

Also, the controller 1100 may detect an available parking space on thebasis of parked vehicles. In detail, when the space between the parkedvehicles is larger than or equal to a certain area, the controller 1100may recognize that the space is an available parking space on the basisof the sensor data. Also, when the space between the parked vehicles islarger than or equal to a certain area even though the parking line 10is not detected, the controller 1100 may recognize that the space is anavailable parking space.

Also, the autonomous driving system 1000 is not limited to theabove-described method and may detect an available parking space on thebasis of the parking line 10 and the parked vehicle.

Also, the autonomous driving system 1000 may generate a map each timethe first vehicle 115 is parked and pulled out.

FIG. 11 is a diagram showing a process of generating, by an autonomousdriving system, a map for pulling out a vehicle according to anembodiment.

Referring to FIG. 11 , the autonomous driving system 1000 may form a mapeach time the first vehicle 115 is parked and pulled out on the basis ofsensor data acquired through at least one sensor placed in the firstvehicle 115.

In this case, the autonomous driving system 1000 may acquire sensor dataregarding surroundings during a first drive and may generate a path fora second drive on the basis of the sensor data acquired during the firstdriving. In detail, the controller 1100 included in the autonomousdriving system 1000 may generate a map of the surroundings of the firstvehicle 115 on the basis of the sensor data acquired during the firstdriving. Also, the controller 1100 may generate a path for the seconddriving on the basis of the map.

Also, when the at least one sensor 1300 is the LiDAR device 1310, thecontroller 1100 may generate a 3D map on the basis of data acquiredthrough the LiDAR device 1310. In detail, the controller 1100 maygenerate the 3D map on the basis of surrounding position informationacquired from the LiDAR device 1310 during the first driving of thefirst vehicle 115. Also, the controller 1100 may generate a path for thesecond driving on the basis of the 3D map.

Also, the autonomous driving system 1000 may include an autonomousparking system. The autonomous parking system may utilize the sensordata. Also, the autonomous parking system may be activated by an inputfrom an occupant. Also, the autonomous parking system may be activatedwhen a parking situation is recognized.

In an embodiment, the autonomous driving system 1000 may implement anautonomous parking system when a vehicle is located in a specific space.For example, when a vehicle is located in a specific space and anoccupant gets out of the vehicle, the autonomous driving system 1000 mayrecognize a situation in which the vehicle is being parked and thusimplement an autonomous parking system.

1.4.2. Autonomous Driving System Using Sensor and Communication 1.4.2.1.Overview

The autonomous driving system 1000 may be implemented using sensor dataacquired from the at least one sensor 1300 and sharing data receivedfrom other devices. The autonomous driving system 1000 may communicatewith other devices through the at least one communication module 1200 toshare data. Also, the autonomous driving system 1000 may use acommunication system to predetermine a risk factor associated with thedriving of the vehicle equipped with the autonomous driving system 1000.

FIG. 12 is a diagram illustrating the type of a communication systemaccording to an embodiment.

Referring to FIG. 12 , an autonomous driving system 1000 may beimplemented through various communication systems. For example, thecommunication system may implement at least one V2X system such as avehicle-to-vehicle (V2V) system, a vehicle-to-infra (V2I) system, avehicle-to-network (V2N) system, a vehicle-to-pedestrian (V2P) system, avehicle-to-cloud (V2C) system, and a vehicle-to-device (V2D) system.

Also, the autonomous driving system 1000 may use at least onestandardized communication system to communicate with other devices. Forexample, the autonomous driving system 1000 may use cellularvehicle-to-everything (C-V2X) and dedicated short-range communication(DSRC) to communicate with other devices, but the present invention isnot limited thereto. In this case, the C-V2X may refer to a 3rdGeneration Partnership Project (3GPP) standard indicating a technologyfor performing V2X communication. Also, the DSRC may refer to a one-wayor two-way short-range wireless communication channel designed for a setof protocols and standards corresponding to vehicles.

1.4.2.1.1. V2V

A first vehicle equipped with the autonomous driving system 1000 maycommunicate with other devices using at least one communication module1200.

Referring to FIG. 12 again, the first vehicle may communicate with othervehicles through a V2V system to share data.

Also, the V2V system may be implemented to transmit or receive sensordata acquired from at least one sensor 1300 included in the firstvehicle to or from other vehicles.

Also, the V2V system may be implemented to transmit or receiveinformation other than the sensor data. For example, the V2V system maybe implemented to transmit a destination of the first vehicle, thenumber of passengers in the first vehicle, the speed of the firstvehicle, and the like, but the present invention is not limited thereto.

Also, for the safety of occupants and passengers, the first vehicle mayuse the V2V system. For example, the first vehicle may receiveinformation regarding a dangerous object present on the path of thefirst vehicle from other vehicles through the V2V system.

1.4.2.1.2. V2I

A first vehicle equipped with the autonomous driving system 1000 maycommunicate with an infrastructure device through at least onecommunication module 1200. In this case, the infrastructure device mayrefer to basic facilities and systems that form an industrial ortransportation base. For example, the infrastructure device may includetraffic lights, speed cameras, road signs, etc., but the presentinvention is not limited thereto.

Also, the infrastructure device may include at least one sensor. Indetail, the infrastructure device may include the at least one sensor inorder to detect a dangerous situation that may happen to vehicles andpedestrians on roads. For example, the at least one sensor may include aLiDAR device, a camera device, etc., but the present invention is notlimited thereto.

Referring to FIG. 12 again, the first vehicle may communicate with theinfrastructure device through a V2I system to share data. Here, theinfrastructure device may be controlled by an external server or mayperform communication to share data without the control of an externalserver.

Also, the V2I system may be implemented to transmit sensor data acquiredfrom at least one sensor included in the first vehicle to theinfrastructure device. Also, the V2I system may be implemented totransmit sensor data acquired from at least one sensor included in theinfrastructure device to the first vehicle.

Also, the V2I system may be implemented to transmit information otherthan the sensor data. In detail, the infrastructure device may transmitregulation information for a space where the infrastructure device isplaced to the first vehicle. For example, the infrastructure device maytransmit information indicating that the space where the infrastructuredevice is placed is a child protection zone to the first vehicle.

Also, when the first vehicle enters a specific zone, the first vehiclemay receive sensor data from the infrastructure device. For example,when the first vehicle enters a child protection zone, the first vehiclemay receive sensor data acquired from an infrastructure device installedin the child protection zone through the V2I system.

1.4.2.1.3. V2C

The first vehicle equipped with the autonomous driving system 1000 maycommunicate with a server through the communication module 1200. In thiscase, the server may be included in a computer of an institution forcontrolling road conditions. For example, the server may include a cloudof a road control system, but the present invention is not limitedthereto. Also, the server may include a local server associated with apredetermined region, a global server for controlling a plurality oflocal servers, etc., but the present invention is not limited thereto.

Referring to FIG. 12 again, the first vehicle may communicate with theserver through the V2C system to share data.

Also, the V2C system may be implemented to transmit sensor data acquiredfrom at least one sensor included in the first vehicle to the server.

Also, the V2C system may be implemented to transmit information otherthan the sensor data.

Also, the first vehicle may receive information regarding an accidentfrom the server. For example, the server may transmit informationindicating that a traffic accident occurred on a path of the firstvehicle to the first vehicle through the V2C system, but the presentinvention is not limited thereto.

Hereinafter, specific embodiments of the autonomous driving system usingsensors and communication will be described in detail.

1.4.2.2. Autonomous Driving System for Safety—Based on Sensors andCommunication

The autonomous driving system 1000 may use sensor data andcommunication-based sharing data in order to protect the safety ofpedestrians and occupants of a vehicle equipped with the autonomousdriving system 1000.

In this case, it will be appreciated that various embodiments describedin Section 1.4.1.2 in which sensor data is used may be applied to anautonomous driving system using sensor data and communication-basedsharing data.

The autonomous driving system 1000 may detect the occurrence of atraffic event through sensors and communication.

FIG. 13 is a diagram showing a situation in which a traffic event hasoccurred in front of a vehicle equipped with an autonomous drivingsystem according to an embodiment.

Referring to FIG. 13 , when a first vehicle 116 acquires sensor dataregarding a traffic event having occurred during driving, the firstvehicle 116 may transmit the sensor data to a server 400 or vehicles 117and 118 associated with the traffic event.

Also, when a traffic event has occurred due to a collision between thesecond vehicle 117 and the third vehicle 118, the vehicles 117 and 118associated with the traffic event may transmit information indicatingthat the traffic event has occurred to the server 400. In this case, theserver 400 may transmit the information indicating that the trafficevent has occurred to the first vehicle 116 located near where thetraffic event has occurred.

Also, the autonomous driving system 1000 may recognize that a vehiclestopped in front of a vehicle equipped with the autonomous drivingsystem is a shared vehicle through communication and may acquireinformation regarding the shared vehicle through communication with theshared vehicle.

For example, a taxi may interfere with the passage of the vehicle whilea passenger gets out of the taxi, and thus the taxi may transmitinformation related to the current situation to the vehicle. Forexample, the taxi may transmit a message indicating that a passenger isgetting out of the vehicle. In this case, the vehicle may determine thatthe taxi is not an illegally parked vehicle through sensor data acquiredfrom at least one sensor and sharing data transmitted from the taxi.

Also, the communication entity is not limited to taxis and may includevarious types of shared vehicles such as buses.

1.4.2.3. Autonomous Driving System for Convenience—Based on Sensors andCommunication

The autonomous driving system 1000 may use sensor data andcommunication-based sharing data in order to provide convenience topedestrians and occupants of a vehicle equipped with the autonomousdriving system 1000.

In this case, it will be appreciated that various embodiments describedin Section 1.4.1.3 in which sensor data is used may be applied to anautonomous driving system using sensor data and communication-basedsharing data.

Also, the autonomous driving system may acquire information regarding anavailable parking space in a parking lot through sensors andcommunication.

FIG. 14 is a diagram showing a situation in which a vehicle equippedwith an autonomous driving system recognizes an available parking spacethrough communication with an infrastructure device in a parking lotaccording to an embodiment.

Referring to FIG. 14 , at least one infrastructure device 700 may beplaced in a parking lot. The at least one infrastructure device 700 mayinclude at least one sensor in order to acquire information regarding anavailable parking space in the parking lot. Also, the infrastructuredevice 700 may store information regarding an available parking spaceincluded in sensor data acquired through a sensor.

Also, when a first vehicle 119 enters the parking lot, theinfrastructure device 700 may transmit the stored information regardingthe available parking space to the first vehicle 119. In this case, acontroller of the first vehicle 119 may move the first vehicle to theavailable parking space on the basis of the information regarding theavailable parking space. In this process, the controller mayadditionally detect a parking space using sensor data obtained through asensor placed in the first vehicle 119. Also, in order to determinewhether the first vehicle 119 can actually park in the available parkingspace at which the first vehicle 119 has arrived according to theinformation regarding the available parking space, the autonomousdriving system 1000 may acquire sensor data regarding the availableparking space using at least one sensor placed in the first vehicle 119.

Also, when a second vehicle 120, which has been parked, exits theparking lot, the second vehicle 120 may transmit information regardingthe space where the second vehicle 120 was parked to the infrastructuredevice 700. In this case, the infrastructure device 700 may recognizethe available parking space by receiving the information regarding thespace where the second vehicle 120 had been parked and storing thereceived information.

Hereinafter, the sensor data and the sharing data will be described.

2. Sensor Data Used by Autonomous Driving System 2.1. Type of Sensor

The autonomous driving system 1000 may include at least one sensor 1300.

Referring to FIG. 2 again, the at least one sensor 1300 may includevarious types of sensors. For example, the at least one sensor 1300 mayinclude at least one LiDAR device 1310, at least one camera device 1320,at least one camera device 1320, at least one radar device 1330, atleast one ultrasonic sensor 1340, etc., but the present invention is notlimited thereto.

2.2. Sensor Data

The autonomous driving system 1000 may acquire sensor data through theat least one sensor 1300. In this case, the sensor data may include rawdata acquirable from the at least one sensor 1300 or data obtained byprocessing the raw data.

Also, the sensor data may include information related to an objectdetected by the at least one sensor 1300. For example, the sensor datamay include position information of the object, distance information ofthe object, shape and/or color information of the object, propertyinformation of the object, etc., but the present invention is notlimited thereto.

Also, the sensor data may include data regarding a single point or dataregarding a plurality of points, which is acquired from the at least onesensor 1300, or processed data which is obtained by processing the dataregarding the single point or the data regarding the plurality ofpoints.

Hereinafter, as a specific example, the sensor data may include a set ofpoint data, point data, a subset of point data, property data, etc.However, the present invention is not limited thereto, and this will bedescribed in detail.

FIG. 15 is a diagram showing a situation in which a vehicle equippedwith an autonomous driving system acquires sensor data regarding anenvironment around the vehicle through at least one sensor according toan embodiment.

For example, when the sensor is the LiDAR device, the sensor data mayinclude point data of each point scanned by the LiDAR device, a set ofpoint data, a subset of point data, property data obtained by processingthe subset of point data, or the like, but the present invention is notlimited thereto.

In this case, the vehicle may detect buildings, vehicles, pedestrians,and the like around the vehicle by using at least one of the point data,the set of point data, the subset of point data, or the property data.

For convenience of description, the following description with referenceto FIGS. 15 to 84 will focus on sensor data of the LiDAR device, but thepresent invention is not limited thereto. It will be appreciated thatsensor data of sensors other than the LiDAR device is applicable toFIGS. 15 to 84 .

FIG. 16 is a diagram showing, on a 3D map, sensor data acquired by theLiDAR device placed in the vehicle of FIG. 15 .

Referring to FIGS. 15 and 16 , the controller included in the autonomousdriving system may form a 3D point data map on the basis of dataacquired from the LiDAR device. In this case, the 3D point data map mayrefer to a 3D point cloud. Also, the sensor data may include dataincluded in the 2D point data map. Also, the position of the origin ofthe 3D point data map may correspond to the optical origin of the LiDARdevice, but the present invention is not limited thereto. The positionof the origin of the 3D point data map may correspond to the position ofthe center of gravity of the LiDAR device or the position or theposition of the center of gravity of the vehicle where the LiDAR deviceis placed.

FIG. 17 is a diagram schematically showing sensor data included in the3D map of FIG. 16 in a 2D plane.

Referring to FIG. 17 , sensor data 2000 may be expressed in a 2D plane.For example, the sensor data may be expressed in the x-z plane, but thepresent invention is not limited thereto.

Also, in the specification, the sensor data may be expressed in the 2Dplane, but this is for schematically representing data on a 3D map.

Also, the sensor data 2000 may be expressed in the form of a data sheet.A plurality of pieces of information included in the sensor data 2000may be expressed as values in the data sheet.

Hereinafter, the sensor data and the meanings of various forms of dataincluded in the sensor data will be described.

2.2.1. Point Data

The sensor data 2000 may include point data. In this case, the pointdata may refer to data that can be primarily acquired when the at leastone sensor 1300 detects an object. Also, the point data may refer to rawdata which is original information acquired from the at least one sensorand which is not processed.

For example, when the sensor is a LiDAR device, the point data maycorrespond to one point included in a point cloud acquired from theLiDAR device.

FIG. 18 is a diagram illustrating point data acquired from at least oneLiDAR device included in an autonomous driving system according to anembodiment.

Referring to FIG. 18 , the LiDAR device may acquire point data 2001 byscanning at least a portion of an object, and the point data 2001 mayinclude position coordinates (x, y, z). Also, in some embodiments, thepoint data 2001 may further include an intensity value I.

In this case, the position coordinates (x, y, z) may be generated basedon information regarding a distance to at least a portion of the object,and the information is acquired by the LiDAR device. In detail, theLiDAR device may compute a distance to at least a portion of the objecton the basis of a time point at which a laser beam is emitted and a timepoint at which a reflected laser beam is received. Also, based on thedistance, the LiDAR device may generate position coordinates of at leasta portion of the object in a Cartesian coordinate system based on theoptical origin of the LiDAR device.

Also, the intensity value I may be generated on the basis of thereflectance of at least a portion of the object acquired by the LiDARdevice. In detail, the magnitude (or strength) of a signal received fromthe LiDAR device varies depending on the reflectance even if the objectis at the same distance. Thus, the LiDAR device may generate anintensity value of at least a portion of the object on the basis of themagnitude (or strength) of the received signal.

Also, the number of pieces of point data 2001 may correspond to thenumber of laser beams emitted from the LiDAR device, scattered by anobject, and then received by the LiDAR device.

More specifically, it is assumed that a laser beam emitted from theLiDAR device is scattered by at least a portion of the object and isreceived by the LiDAR device. Each time the laser beam is received, theLiDAR device may process a signal corresponding to the received laserbeam to generate the point data 2001.

However, the present invention is not limited thereto, and when thesensor is a camera device, the sensor data 2000 may include the pointdata 2001.

In this case, the point data 2001 may correspond to one pixel acquiredfrom the camera device. In detail, the point data 2001 may correspond toone pixel acquired through an RGB sensor included in the camera device.For example, when a plurality of pixels are present in a light-receivingunit of a camera, the point data 2001 may be generated for each pixel,and the point data 2001 may include pixel values (e.g., RGB color valuesin the case of an RGB sensor) of the pixels and position information ofan object corresponding to the position of the pixels.

Also, the point data 2001 may include shape and/or color informationacquired from the camera device.

However, the present invention is not limited thereto, and when thesensor is a radar device, the point data 2001 may correspond to onepoint acquired from the radar device. In detail, the point data 2001 mayinclude position coordinates acquired from the radar device.

For example, in the case of a radar, a plurality of Tx antennas transmita plurality of radio waves, and a plurality of Rx antennas receive aplurality of radio waves which are scattered by an object and thenreturned. In this case, the radar may acquire position information ofthe object with respect to the plurality of received radio waves, andthe point data 2001 may indicate the position information of the objectwith respect to one of the plurality of radio waves.

2.2.2. Set of Point Data

The sensor data may include a set of point data 2100. In this case, theset of point data 2100 may include multiple pieces of point data 2001.Also, the set of point data 2100 may be included in one frame. In someembodiments, the set of point data 2100 may be included in multipleframes.

For example, when the sensor is a LiDAR device, the sensor data mayinclude the set of point data 2100, and the set of point data 2100 maycorrespond to a point cloud of one frame acquired from the LiDAR device.

FIG. 19 is a diagram illustrating a set of point data acquired from theLiDAR device included in the vehicle of FIG. 16 .

Referring to FIG. 19 , the set of point data 2100 shown in FIG. 19 maybe acquired from the LiDAR device.

Also, the set of point data 2100 may refer to a plurality of pieces ofpoint data that are generated when the LiDAR device scans the field ofview of the LiDAR device once. For example, when the horizontal field ofview of the LiDAR device is 180 degrees, the set of point data 2100 mayrefer to all point data acquired when the LiDAR device scans 180 degreesonce.

Also, the set of point data 2100 may include the position coordinates(x, y, z) and intensity value I of an object present in the field ofview of the LiDAR device. Also, the position coordinates (x, y, z) andintensity value I of the point data 2001 included in the set of pointdata 2100 may be expressed in a data sheet.

Also, the set of point data 2100 may include noise data. The noise datamay be generated by an external environment regardless of the objectlocated in the field of view of the LiDAR device. For example, the noisedata may include noise due to interference between LiDARs, noise due toambient light such as sunlight, noise due to an object outside ameasurable range, etc., but the present invention is not limitedthereto.

Also, the set of point data 2100 may include background information. Thebackground information may refer to at least one piece of point data notrelated to an object among a plurality of pieces of point data includedin the set of point data 2100. Also, the background information may beprestored in the autonomous driving system including the LiDAR device.For example, the background information may include information on aimmovable object such as a building (or an stationary object located ata fixed position) and may be prestored in the autonomous driving systemincluding the LiDAR device in the form of a map.

However, the present invention is not limited thereto, and even when thesensor is a camera device, the sensor data 2000 may include the set ofpoint data 2100. In this case, the set of point data 2100 may correspondto one frame acquired from the camera device. Also, the set of pointdata 2100 may correspond to all pixels which are acquired from thecamera device and which are in the field of view of the camera device.In detail, the camera device may generate a set of point data 210 of oneframe representing shape and/or color information of objects present inthe field of view of the camera device by photographing thesurroundings.

For example, when a plurality of pixels are present in a light-receivingunit of a camera, the set of point data 2100 may include a plurality ofpieces of point data 2001 generated for each of the plurality of pixels.

However, the present invention is not limited thereto, and even when thesensor is a radar device, the sensor data 2000 may include the set ofpoint data 2100. In this case, the set of point data 2100 may includethe position coordinates of all the objects which are acquired from theradar device and which are in the field of view of the radar device.

For example, the set of point data 2100 may include a plurality ofpieces of point data corresponding to a plurality of received radiowaves.

2.2.3. Sub Set of Point Data

Referring to FIG. 19 again, the sensor data 2000 may include a subset ofpoint data 2110. In this case, the subset of point data 2110 may referto a plurality of pieces of point data that represent the same object.For example, when the set of point data 2100 includes a plurality ofpieces of point data that represent a vehicle, the plurality of piecesof point data may constitute one subset of point data 2110.

Also, the subset of point data 2100 may be included in the set of pointdata 2100. Also, the subset of point data 2100 may refer to at least oneobject included in the set of point data 2100 or at least a portion ofthe object. In detail, the subset of point data 2110 may refer to aplurality of pieces of point data that represents a first object amongthe plurality of pieces of point data included in the set of point data2100.

Also, the subset of point data 2110 may be acquired by clustering atleast one piece of point data related to a dynamic object among theplurality of pieces of point data included in the set of point data2100. In detail, the subset of point data 2110 may be acquired bydetecting a immovable object and a dynamic object (or a movable object)included in the set of point data 2100 using the background informationand then by grouping data related to one object into a certain cluster.

Also, the subset of point data 2110 may be generated using machinelearning. For example, the controller 1100 may determine that at leastsome of the plurality of pieces of data included in the sensor data 2000represent the same object on the basis of machine learning performed onvarious objects.

Also, the subset of point data 2110 may be generated by segmenting theset of point data 2100. In this case, the controller 1100 may segmentthe set of point data 2100 in units of a predetermined segment. Also, atleast one segment unit of the segmented set of point data may refer toat least a portion of the first object included in the set of point data2100. Also, a plurality of segment units representing the first objectmay correspond to the subset of point data 2110.

For example, when the sensor is a LiDAR device, the subset of point data2110 may correspond to a plurality of pieces of point data related tothe first object included in the set of point data 2100 acquired fromthe LiDAR device.

FIG. 20 is a diagram illustrating a subset of point data acquired fromat least one LiDAR device included in an autonomous driving systemaccording to an embodiment.

Referring to FIG. 20 , the set of point data 2020 may include aplurality of subsets of point data 2110, 2120, 2130, 2140, and 2150.

The plurality of subsets of point data 2110, 2120, 2130, 2140, and 2150may include a plurality of pieces of point data representing at least aportion of an object. Here, the controller 1100 may determine that theplurality of pieces of point data 2001 represent at least a portion ofthe same object on the basis of the position coordinates (x, y, z) andthe intensity value I of the plurality of pieces of point data 2001.Accordingly, the controller 1100 may define the plurality of pieces ofpoint data 2001 as a subset of point data and generate property data ofthe object on the basis of the subset of point data.

For example, a first subset of point data 2110 may represent at least aportion of “human,” a second subset of point data 2120 may represent atleast a portion of “vehicle,” a third subset of point data 2130 mayrepresent at least a portion of “center line,” a fourth subset of pointdata 2140 may represent at least a portion of “road shoulder line,” afifth subset of point data 2150 may represent at least a portion of“lane line,” and a sixth subset of point data 2160 may represent atleast a portion of “building,” but the present invention is not limitedthereto.

In this case, the first subset of point data 2110 may refer to at leasta portion of the same “human.” In detail, the first subset of point data2110 may include the position coordinates (x, y, z) and the intensityvalues I of the plurality of pieces of point data included in the firstsubset of point data 2110. In this case, the plurality of pieces ofpoint data may constitute one subset of point data representing at leasta portion of “human.”

2.2.4. Property Data

The sensor data 2000 may include property data 2200. In this case, theproperty data 2200 may be determined based on at least one subset ofpoint data 2110. In detail, the property data 2200 may includeinformation regarding various properties, such as type, size, speed, anddirection, of an object which are represented by the at least one subsetof point data 2110. Also, the property data 2200 may be data obtained byprocessing at least a portion of the at least one subset of point data2110.

For example, when the sensor is a LiDAR device, the sensor data 2000 mayinclude property data (see reference number 2200 of FIG. 21 ), and theproperty data may be generated based on the subset of point data 2110included in the set of point data 2100 acquired from the LiDAR device.

Also, a process of generating the property data 2200 on the basis of thesubset of point data 2110 included in the set of point data 2100 may usea point cloud library (PCL) algorithm.

As an example, a first process related to the generation of the propertydata 2200 using the PCL algorithm may include operations ofpreprocessing a set of point data, removing background information,detecting feature (key) points, defining a descriptor, matching thefeature points, and estimating the property of an object, but thepresent invention is not limited thereto.

In this case, the operation of preprocessing a set of point data mayrefer to the processing of the set of point data into a form suitablefor the PCL algorithm. In the first process, point data that is includedin the set of point data 2100 and that is not related to the extractionof property data may be removed. For example, the operation ofpreprocessing data may include operations of removing noise dataincluded in the set of point data 2100 and re-sampling a plurality ofpieces of point data included in the set of point data 2100, but thepresent invention is not limited thereto.

Also, through the operation of removing background information, in thefirst process, the subset of point data 2110 related to the object maybe extracted by removing the background information included in the setof point data 2100.

Also, through the operation of detecting feature points, in the firstprocess, a feature point suitably representing the shape characteristicsof the object may be detected among a plurality of pieces of point dataincluded in the subset of point data 2110 related to the object, whichremains after the background information is removed.

Also, through the operation of defining the descriptor, in the firstprocess, a descriptor for describing a characteristic unique to each ofthe detected feature points may be defined.

Also, through the operation of matching the feature points, in the firstprocess, corresponding feature points may be chosen by comparing adescriptor of feature points included in prestored template data relatedto the object and a descriptor of feature points of the subset of pointdata 2110.

Also, through the operation of estimating the property of an object, inthe first process, the property data 2200 may be generated by detectingan object represented by the subset of point data 2110 using a geometricrelationship of the chosen feature points.

As another example, a second process related to the generation of theproperty data 2200 may include operations of preprocessing data,detecting data regarding an object, clustering the data regarding theobject, classifying the clustered data, tracking the object, etc., butthe present invention is not limited thereto.

In this case, through the operation of detecting data regarding anobject, in the second process, a plurality of pieces of point datarepresenting an object among a plurality of pieces of point dataincluded in the set of point data 2100 may be extracted using prestoredbackground data.

Also, through the operation of clustering the data regarding the object,in the second process, a subset of point data 2110 may be extracted byclustering at least one piece of point data representing one objectamong the plurality of pieces of point data.

Also, through the process of classifying the clustered data, in thesecond process, the class information of the subset of point data 2110may be classified or determined using a machine learning model or a deeplearning module which is pre-learned.

Also, through the operation of tracking the object, in the secondprocess, the property data 2200 may be generated based on the subset ofpoint data 2110. For example, a controller that performs the secondprocess may display the position of the object using the center positioncoordinates and volume of the plurality of subsets of point data 2110.Accordingly, the controller may estimate the movement direction andspeed of the object by defining a correspondence relationship based oninformation on the similarity in distance and shape between a pluralityof subsets of point data acquired from successive frames and then bytracking the object.

FIG. 21 is a diagram illustrating property data generated from a subsetof point data acquired from a LiDAR device included in an autonomousdriving system according to an embodiment.

Referring to FIG. 21 , the property data 2200 may be generated for eachpoint data 2001 included in the subset of point data 2110. In detail,the property data 2200 may be assigned to each piece of point data 2001included in the subset of point data representing at least a portion ofone object.

For example, the property data 2200 of the subset of point data may begenerated for each piece of point data 2001 included in the subset ofpoint data 2110 representing at least a portion of a human. In thiscase, the property data 2200 may include a class information, a centerposition information, a size information of the like of the human, butthe present invention is not limited thereto. A plurality of pieces ofinformation included in the property data will be described in detailbelow.

FIG. 22 is a diagram showing another example of property data of FIG. 21.

Referring to FIG. 22 , the property data 2200 may be generated in commonfor a plurality of pieces of point data included in the subset of pointdata 2110. That is, one piece of property data 2200 may be generated forone subset of point data 2110 representing at least a portion of oneobject.

For example, when the object is a human, one piece of property data maybe generated for a plurality of pieces of point data included in asubset of point data representing at least a portion of the human.

FIG. 23 is a diagram illustrating a plurality of pieces of informationincluded in property data according to an embodiment.

Referring to FIG. 23 , the property data 2200 may include a classinformation 2210, a center position information 2220, a size information2230, a shape information 2240, a movement information 2250, anidentification information 2260 of the like of the object which arerepresented by the subset of point data 2110, but the present inventionis not limited thereto.

Hereinafter, a plurality of pieces of information included in theproperty data 2200 will be described in detail.

The property data 2200 may include a class information 2210 indicatingthe class of the object represented by the subset of point data 2110.

FIG. 24 is a diagram illustrating a class information included inproperty data according to an embodiment.

Referring to FIG. 24 , the class information 2210 may include a classrelated to the type of the object, a class related to the type of aportion of the object, a class related to a situation of a regionincluding the object, etc., but the present invention is not limitedthereto.

In this case, the class information 2210 may be associated with the typeof the object represented by the subset of point data 2110.

In this case, the class information related to the type of the objectmay be determined depending on the kind of the object. For example, whenthe object is a human, the class information of the subset of point datamay be determined as “human,” but the present invention is not limitedthereto. The class information 2210 may be determined as a lower classof the human. As a specific example, when the object is a male, theclass information 2210 of the subset of point data may be determined as“male.” Also, the lower class of the human may include “female,”“child,” “the elderly,” “pedestrian,” etc., but the present invention isnot limited thereto.

Also, the class information 2210 may be associated with the type of aportion of the object. In detail, fora class related to the type of aportion of the object, when the set of point data 2100 includes thesubset of point data 2110 representing a portion of the object, thecontroller 1100 may determine that the subset of point data 2110represents a portion of the object. For example, when the subset ofpoint data 2110 represents a human arm, the class information 2210 ofthe subset of point data may be determined as “human” or may bedetermined as “human arm.”

Also, the class information 2210 may be associated with the situation ofthe region including the object. In this case, the class related to thesituation of the region including the object may be determined based ona plurality of subsets of point data. In detail, the controller 1100 maydetermine the class information 2210 of the object on the basis of thesubset of point data representing at least a portion of the object, andthe controller 1100 may determine class information related to thesituation of the region including the object in consideration of both ofthe subset of point data 2110 and another plurality of subsets of pointdata.

As a specific example, when a LiDAR device acquires a plurality ofsubsets of point data representing at least a portion of a worker and anexcavator that are working at a construction site, the controller 1100may determine that the class information of the worker and the excavatoris “construction site” on the basis of the plurality of subsets of pointdata.

Also, the class information 2210 may be determined based on a lookuptable prestored in the autonomous driving system 1000.

More specifically, the autonomous driving system 1000 may generate andstore a lookup table that matches objects to the class information 2210of the objects. In this case, the controller 1100 may determine theclass information 2210 of the subset of point data on the basis of thelookup table.

In this case, the lookup table may be used to determine a class relatedto the situation of the region including the object. For example, thelookup table may match the class information of a plurality of objectsto a class related to a situation of a region including the plurality ofobjects. As a specific example, when the class information of theplurality of objects includes at least a portion some of “worker,”“excavator,” and “construction sign,” the lookup table may match theplurality of pieces of class information to “construction site,” whichis a class related to the situation of the region including theplurality of objects. In this case, the controller may determine thatthe class of the plurality of objects is “construction site” using thelookup table.

Also, the class information 2210 may be determined using machinelearning. In detail, the autonomous driving system 1000 may pre-learn acorrespondence relationship by repeatedly matching the subset of pointdata 2110 to an object represented by the subset of point data and maydetermine the class information 2210 of the object on the basis of thecorrespondence relationship.

Also, the class information 2210 may include at least one class. As anexample, the controller may determine the class information 2210 of thesubset of point data as one class (e.g., “human”). Also, as anotherexample, a plurality of classes (e.g., “human” and “construction site”)instead of one class may be included in the class information 2210.

Also, the class information 2210 may include a class group including atleast one class. Here, the class group may refer to a collection ofclasses having similar or common characteristics. In this case, theclass group may be preset and stored by a controller or a user, but thepresent invention is not limited thereto.

As an example, classes such as “human,” “vehicle registration plate,”and “identity document” have a common characteristic in that the classesare related to personal information and thus may constitute a classgroup related to the personal information. As another example, classessuch as “human” and “vehicle” have a common characteristic in that theclasses are related to a movable object and thus may constitute a classgroup related to the movable object.

Also, referring to FIG. 23 , the property data 2200 may include a centerposition information 2220 of the subset of point data.

FIG. 25 is a diagram illustrating a center position information includedin property data according to an embodiment.

Referring to FIG. 25 , the center position information 2220 may becomputed based on a subset of point data 2110 representing at least aportion of an object included in the set of point data 2100. Forexample, the center position information 2220 may refer to the positioncoordinates (x, y, z) and center position coordinates (x_(o), y_(o),z_(o)) of each of a plurality of pieces of the point data included inthe subset of point data 2110. In this case, the center positioncoordinates (x_(o), y_(o), z_(o)) may be coordinates indicating theaverage of the position coordinates (x, y, z) of the plurality of piecesof point data, but a method of computing the center position coordinates(x_(o), y_(o), z_(o)) is not limited thereto and may be used in variousways.

Also, the center position information 2220 may be expressed in acoordinate system with at least one reference position as the origin.For example, the reference position may include the position of a LiDARdevice configured to acquire point data, the position of an apparatusincluding the LiDAR device, and the like, and the center positioninformation 2220 may be expressed in a coordinate system with thereference position as the origin, but the present invention is notlimited thereto. The coordinate system and the origin, which serves as areference, will be described in detail below.

Also, referring to FIG. 23 , the property data 2200 may include a sizeinformation 2230 of the sub set of point data.

FIG. 26 is a diagram illustrating a size information included inproperty data according to an embodiment.

Referring to FIG. 26 , the size information 2230 may correspond to thesize of an object represented by the subset of point data 2110.

In this case, the size information 2230 may be computed based on thesubset of point data 2110 indicating at least a portion of an objectincluded in the set of point data 2100. For example, the sizeinformation 2230 may be computed based on a volume that the subset ofpoint data 2110 occupies in the set of point data 2100. In detail, thecontroller 1100 may extract a space that the subset of point data 2110occupies in the set of point data 2100 and may compute size information2230 of the object by computing the volume of the extracted space.

Also, the size information 2230 may be computed based on positioninformation of the plurality of pieces of point data included in thesubset of point data 2110. In detail, since the plurality of pieces ofpoint data represent the surface of the object, the size information2230 may be acquired by computing the volume of the object using theposition information of the point data representing the surface of theobject.

Also, the size information 2230 may be computed based on the centerposition information 2220 and the subset of point data 2110. Forexample, the size information 2230 may be generated by computing thevolume of a rectangular parallelepiped shape having a center at thecenter position coordinates (x_(o), y_(o), z_(o)) included in the centerposition information 2220 and having a width, a length, and a heightcorresponding to the width, length and height of the subset of pointdata 2110. It will be appreciated that the size information 2230 may becomputed by computing the volume of various shapes such as not only arectangular parallelepiped but also a cube, a polyhedron, a sphere, andan ellipse.

Also, referring to FIG. 23 , the property data 2200 may include a shapeinformation 2240 of the subset of point data.

In this case, the shape information 2240 may indicate the shape of theobject represented by the subset of point data 2110. Here, the shape ofthe object may include the actual shape of the object and may alsoinclude a processed shape that is expressed by processing the shape ofthe object. Here, the processed shape may include a similar shape thatis expressed as being similar to the actual shape of the object and anarbitrary shape that is different from the actual shape of the objectbut indicates the presence of the object.

For example, the shape information 2240 may include a templateinformation 2241 in which the object is represented using apredetermined shape when representing the arbitrary shape and mayinclude a skeleton information 2242 in which the object is representedusing a predetermined number of points or less when representing thesimilar shape, but the present invention is not limited thereto.

FIG. 27 is a diagram illustrating a template information of shapeinformation included in property data according to an embodiment.

Referring to FIG. 27 , the template information 2241 may represent anobject represented by the subset of point data 2110 using apredetermined shape. In detail, the template information 2241 mayindicate a predetermined shape corresponding to the class information2210 on the basis of the class information of the subset of point data.For example, when the class information 2210 of the subset of point datais related to a human, the template information 2241 may correspond to apredetermined shape having a human shape, but the present invention isnot limited thereto.

Also, the template information 2241 may be prestored in the autonomousdriving system 1000. In detail, the autonomous driving system 1000 mayprestore the template information 2241 corresponding to the classinformation 2210 of the object or acquire from an external server.

FIG. 28 is a diagram illustrating a skeleton information of shapeinformation included in property data according to an embodiment.

Referring to FIG. 28 , the skeleton information 2242 may represent anobject represented by the subset of point data 2110 using apredetermined number or less of points. In detail, the skeletoninformation 2242 may represent the shape of the object using the minimumnumber of points capable of expressing the shape of the object on thebasis of the class information 2210 of the subset of point data. Forexample, when the class information of the subset of point data isrelated to a human, the skeleton information may correspond to aplurality of points corresponding to a human joint, but the presentinvention is not limited thereto.

Also, referring to FIG. 23 , the property data 2200 may include amovement information 2250 of the subset of point data. In this case, themovement information 2250 may include the movement direction, speed,tracking information, and the like of the object represented by thesubset of point data 2110, but the present invention is not limitedthereto.

Also, the movement information 2250 may be generated by defining acorrespondence relationship between the positions of the same object insuccessive frames. Here, defining the correspondence relationshipbetween the positions of the same object in successive frames meansspecifying the same object in each of the successive frames, acquiringposition information of the specified object, and associating theacquired position information with a position of the specified objectwith time.

For example, the movement information 2250 may be generated by thecontroller 1100 through a predetermined algorithm. The algorithm mayinclude acquiring a first set of point data corresponding to a firstframe of at least one sensor, acquiring a second set of point datacorresponding to a second frame following the first frame, extracting afirst subset of point data representing a first object included in thefirst set of point data, extracting a second subset of point datarepresenting the first object included in the second set of point data,defining a correspondence relationship between the subsets of point dataon the basis of similarity in distance or shape between the first subsetof point data and the second subset of point data, and generating amovement direction, speed, and the like of the first object on the basisof position information of the subsets of point data, but the presentinvention is not limited thereto.

Also, by accumulating the movement directions and speeds of the firstobject which are generated for a plurality of frames, the controller1100 may generate tracking information of the first object.

Also, referring to FIG. 23 , the property data 2200 may include anidentification information 2260 of the subset of point data. In thiscase, the identification information 2260 may be generated todistinguish the subset of point data 2110 from other sets of point data.

Also, the identification information 2260 may be generated to expressthat a plurality of pieces of point data included in the subset of pointdata 2110 represent the same object. In detail, the identificationinformation 2260 may include a common ID of the subset of point data2110. Also, the ID may be generated for each of a plurality of pieces ofpoint data included in the subset of point data 2110. In this case, theID may be expressed with at least one serial number, but the presentinvention is not limited thereto.

Hereinafter, a method of the autonomous driving system 1000 controllinga vehicle using the sensor data 2000 will be described.

2.3. Vehicle Control Using Sensor Data

The controller 1100 included in the vehicle equipped with the autonomousdriving system 1000 may control the vehicle using sensor data acquiredfrom the at least one sensor 1300. For example, the controller 1100 maymatch the sensor data to a high-precision map (or a high-definition (HD)map), control the direction and speed of the vehicle, or control thepath of the vehicle, but the present invention is not limited thereto.Here, the high-definition map refers to a map in which a immovableobject or a dynamic object is shown with high precision (e.g., precisionat the level of a centimeter) for driving a vehicle and may be expressedin 2D or 3D.

Hereinafter, a specific embodiment of vehicle control using the sensordata will be described.

2.3.1. Matching of Sensor Data to High-Definition Map

The controller 1100 included in the autonomous driving system 1000 mayupdate a high-definition map by matching sensor data 2000 to the map. Indetail, the controller 1100 may match position information of at leastone object acquired from the at least one sensor 1300 to ahigh-definition map 1420 downloaded from the outside. Here, details onhow to generate the high-definition map 1420 have been described inSection 1.3.5, and thus will be omitted here.

FIG. 29 is a diagram showing that an autonomous driving system matches asubset of point data acquired from a sensor to a high-definition mapaccording to an embodiment.

Referring to FIG. 29 , the controller 1100 may match a plurality ofsubsets of point data 2110 and 2120 acquired from the at least onesensor 1300 to the high-definition map 1420 and then display thematching result. In detail, the controller 1100 may compare positioninformation included in the plurality of subsets of point data 2110 and2120 to position information of environments surrounding the pluralityof subsets of point data in the high-definition map 1420, match theplurality of subsets of point data 2110 and 2120 to the high-definitionmap 1420, and display the matching result.

For example, the controller 1100 may match a first subset of point datarepresenting at least a portion of a human and a second subset of pointdata representing at least a portion of a vehicle to the high-definitionmap 1420.

FIG. 30 is a diagram showing that an autonomous driving system matchesproperty data of an object to a high-definition map according to anembodiment.

Referring to FIG. 30 , the controller 1100 may match a plurality ofpieces of property data 2201 and 2202 generated based on the pluralityof subsets of point data 2110 and 2120 to the high-definition map 1420and display the matching result.

More specifically, the controller 1100 may generate the plurality ofpieces of property data 2201 and 2202 without matching the plurality ofsubsets of point data 2110 and 2120 acquired from the at least onesensor 1300 to the high-definition map 1420. In this case, thecontroller 1100 may match the plurality of pieces of property data 2201and 2202 to the high-definition map 1420 and display the matchingresult.

For example, the controller 1100 may generate first property data 2201on the basis of the first subset of point data 2110 representing atleast a portion of a human and generate second property data 2202 on thebasis of the second subset of point data 2120 representing at least aportion of a vehicle. Here, the first property data 2201 includes shapeinformation of the human, and the second property data 2202 includesshape information of the vehicle. Thus, the controller 1100 may matchthe plurality of pieces of shape information to the high-definition map1420 and display the matching result.

Also, the plurality of pieces of property data 2201 and 2202 are notlimited to the shape information and may refer to various pieces ofinformation included in the property data such as center positioninformation and size information.

Also, the controller may control a vehicle using the high-definition map1420 to which the plurality of subsets of point data 2110 and 2120 orthe plurality of pieces of property data 2201 and 2202 are matched. Forexample, the controller may determine whether an obstacle is present onthe path of the vehicle on the basis of the high-definition map 1420 andmay control the speed, direction, or path of the vehicle according tothe determination.

2.3.2. Control of Direction and Speed of Vehicle

Also, the controller 1100 included in the autonomous driving system 1000may control the direction and speed of the vehicle equipped with theautonomous driving system 1000 using the sensor data 2000. In detail,when an obstacle is found on the path of the vehicle through the atleast one sensor, the controller 1100 may control the direction andspeed of the vehicle in order to avoid the corresponding obstacle. Forexample, when a pedestrian is detected on the path of the vehicle, thecontroller 1100 may stop the vehicle or control a steering device tochange the direction of the vehicle in order to avoid the pedestrian.

2.3.3. Path Control for Vehicle

Also, the controller 1100 included in the autonomous driving system 1000may control the path of the vehicle using the sensor data 2000.

FIG. 31 is a diagram showing a situation in which an autonomous drivingsystem changes a path to avoid an obstacle obstructing the driving of avehicle according to an embodiment.

Referring to FIG. 31 , when the movement of a pedestrian is detected onthe driving path of a vehicle 121 equipped with the autonomous drivingsystem 1000, the controller 1100 may change the path of the vehicle 121in order to avoid the pedestrian.

In detail, the controller 1100 may stop the vehicle in order to avoid acollision between the vehicle 121 and the pedestrian. However, thepresent invention is not limited thereto, and the controller 1100 maymodify the path of the vehicle so that the vehicle can travel away fromthe pedestrian. Vehicle path planning will be described in detail below(in Section 5.2.2.2.).

3. Data Sharing System

A data sharing system according to an embodiment may include a firstdevice and a second device, each of which includes a communicationmodule. Also, the first device may share data with the second device. Inthis case, the type of sharing data is not limited and may includesensor data.

For example, a vehicle equipped with an autonomous driving system mayshare data with other devices using the data sharing system in order toavoid a risk that may occur during the driving of the vehicle.

3.1. Data Sharing Entity

A device including at least one communication module may be a datasharing entity. In detail, the data sharing entity may be a transmissionentity that transmits data or a reception entity that receives data.Also, the data sharing entity may include a vehicle, an infrastructuredevice, a server, etc., but the present invention is not limitedthereto. Also, the data sharing entity may include a plurality ofsensors included in one device or a plurality of sensors included indifferent devices.

FIG. 32 is a diagram showing a situation in which data is shared betweena plurality of devices according to an embodiment.

Referring to FIG. 32 , a plurality of devices 100, 400, and 700 mayshare data with each other. In this case, the plurality of devices 100,400, and 700 may include at least one communication module 1200 toperform communication. In this case, the plurality of devices mayinclude a vehicle 100, an infrastructure device 700, a server (cloud)400, a mobile device, etc., but the present invention is not limitedthereto.

For example, the vehicle 100 may share data with other devices through aV2V system. Also, the vehicle 100 may share data with the infrastructuredevice 700 through a V2I system. Also, the vehicle 100 may share datawith the server 400 through a V2C system.

In this case, the vehicle 100 may transmit sensor data 2000 acquiredfrom at least one sensor 1300 included in the vehicle 100 to anothervehicle, the infrastructure device 700, or the server 400. Also, thevehicle 100 may receive sensor data from the other vehicle, theinfrastructure device 700, or the server 400.

3.2. Data Sharing Time

Also, data sharing between a plurality of devices each including atleast one communication module may be performed at different timesdepending on the situation. For example, the time of data sharingbetween the plurality of devices may include a communication start timepoint, a specific-event occurrence time point, or the like, but thepresent invention is not limited thereto.

As a specific example, the time of data sharing between a first deviceand a second device may correspond to a start time point ofcommunication between the first device and the second device. In thiscase, when the distance between the first device and the second devicereaches an available communication distance, the first device and thesecond device may start communication and may share data when thecommunication is started.

As another example, the data sharing between the first device and thesecond device may be performed when the first device is located within acertain range from the second device. In this case, the certain rangemay be different from the available communication distance and may bepreset by controllers of the first device and the second device or anexternal server.

As still another example, the data sharing between the first device andthe second device may be performed when an event related to the firstdevice occurs. In detail, it is assumed that an accident occurs inrelation to the first device, upon the occurrence of the accident, thesecond device may transmit data related to the accident to the firstdevice.

As yet another example, the data sharing between the first device andthe second device may be performed when the first device receives a datarequest message from the second device. In detail, the second device maytransmit a data request message to the first device, and the firstdevice may transmit data to the second device in response to the requestmessage.

As yet another example, the data sharing between the first device andthe second device may be performed when the first device gains approvalfor data transmission from an external server. In detail, the firstdevice may obtain permission for transmission of data related topersonal information from an external server before transmitting thedata related to the personal information, and the first device maytransmit the data to the second device when the external server approvesdata transmission.

As yet another example, the data sharing between the first device andthe second device may be performed when the first device enters aspecific region. In detail, when the first device enters a specificregulation region such as a child protection zone, the second device maytransmit data related to the specific region to the first device.

As yet another example, the data sharing between the first device andthe second device may be performed when a user of the first deviceenters an input related to data sharing. In detail, when the firstdevice receives an input for sharing data with the second device from auser who is in the first device, the first device and the second devicemay transmit or receive data.

Hereinafter, sharing data transmitted or received when data is sharedwill be described in detail.

3.3. Sharing Data 3.3.1. Definition of Sharing Data

In the specification, sharing data 3000 may be defined as a conceptincluding all sharing data when the data is shared between two or moredevices. In this case, a first device may transmit the sharing data 3000to a second device. Also, the first device may receive the sharing data3000 from the second device.

For example, the sharing data 3000 may include sensor data acquiredthrough a sensor placed in the first device, but the present inventionis not limited thereto.

3.3.2. Content of Sharing Data

The content of the sharing data may be understood as a concept includingthe content or type of data included in the sharing data 3000. In otherwords, the content of the sharing data forms the sharing data 3000, andthe sharing data 3000 is determined according to the type of the dataincluded in the content of the sharing data.

FIG. 33 is a diagram showing the content types of sharing data which maybe included in the sharing data according to an embodiment.

FIG. 34 is a diagram specifically showing the content of the sharingdata of FIG. 33 .

Referring to FIGS. 33 and 34 , the sharing data 3000 may include varioustypes of data as content.

For example, the content of the sharing data may include sensor data2000 acquired from at least one sensor. In other words, a controllerincluded in the first device may generate sharing data 3000 on the basisof the sensor data 2000. In this case, the content of the sharing datamay include a set of point data 3100, point data 3101, a subset of pointdata 3110, property data 3200, privacy protection data 3300, or thelike, but the present invention is not limited thereto. In this case,the privacy protection data 3300 will be described in detail below.

Also, the content of the sharing data may include other data includinginformation regarding a data sharing entity. For example, a vehicleincluding the at least one communication module 1200 may share thesharing data 3000 including information regarding the vehicle with otherdevices. For example, the content of the sharing data may include theother data 3400 in addition to the sensor data 2000, and the other data3400 may include the destination, speed, and size of the vehicle, thenumber of occupants in the vehicle, etc., but the present invention isnot limited thereto.

3.4. Processing of Received Sharing Data

A device which has received the sharing data 3000 may generate variouspieces of information using the sensor data 2000 and the sharing data3000. For example, a device which has received the sharing data 3000 mayrecognize an object represented by the sensor data 2000 and the sharingdata 3000 using the sensor data 2000 and the sharing data 3000.

FIG. 35 is a diagram showing a situation in which sensor data is sharedbetween a vehicle and an infrastructure device.

Referring to FIG. 35 , a first vehicle 122 and an infrastructure device700 may share sensor data acquired through at least one sensor (e.g., aLiDAR device) each included in the first vehicle 122 and theinfrastructure device 700.

3.4.1. Method of Processing Received Sharing Data According to Type

Referring to FIG. 35 again, the infrastructure device 700 may transmitsharing data 3000 including sensor data acquired through at least onesensor to the first vehicle 122.

For example, the infrastructure device 700 may transmit sharing dataincluding a set of point data or transmit sharing data 3000 includingproperty data. However, the present invention is not limited thereto,and the content of the sharing data may or may not include both of theset of point data and the property data. In this case, the first vehicle122 may process the sharing data 3000 in different manners depending onthe type of content of the sharing data.

Hereinafter, embodiments in which the first vehicle 122 processes thesharing data 3000 when the infrastructure device 700 transmits a set ofpoint data and when the infrastructure device 700 transmits propertydata will be described.

3.4.1.1. Case of Transmitting Set of Point Data

FIG. 36 is a diagram illustrating a situation in which a set of pointdata is included in the content of sharing data according to anembodiment.

Referring to FIG. 36 , the infrastructure device 700 may transmitsharing data 3000 including a first set of point data 3100 acquired froma sensor to the first vehicle 122. In this case, the first set of pointdata 3100 may include a first subset of point data 3110 representing atleast a portion of a second vehicle 123 and a second subset of pointdata 3120 representing at least a portion of a pedestrian 800.

Also, the first vehicle 122 may acquire a second set of point data 2100through at least one sensor. In this case, the second set of point data2100 may include a third subset of point data 2110 representing at leasta portion of the second vehicle 123. Also, the pedestrian 800 who islocated in the field of view of the sensor of the first vehicle 122 iscovered by the second vehicle 123, and thus the second set of point data2100 may not include a subset of point data representing at least aportion of the pedestrian 800.

Also, through the data sharing system according to an embodiment, thefirst vehicle 122 may acquire information regarding an object that isnot included in the sensor data. For example, when the first vehicle 122cannot acquire sensor data regarding the pedestrian 800 through at leastone sensor, the first vehicle 122 cannot recognize the pedestrian 800,which may cause an unexpected accident related to the first vehicle 122.In order to prevent the above situation, the infrastructure device 700may share sensor data related to the pedestrian 800, which cannot beacquired by the first vehicle 122, with the first vehicle 122.

FIG. 37 is a diagram illustrating a method of processing, by a firstvehicle, a shared first set of point data and a second set of point dataaccording to an embodiment.

Referring to FIG. 37 , a controller of the first vehicle 122 mayrecognize at least one object included in the field of view of a sensorof the first vehicle 122 using a second set of point data 2100 and ashared first set of point data 3100.

In detail, a controller 1100 included in the first vehicle 122 maygenerate third property data 2201 on the basis of a third subset ofpoint data 2110 included in the second set of point data 2100.

Here, the property data 2201 may include a class information, a centerposition information, a size information, etc. of the second vehicle 123which are represented by the third subset of point data 2110, but thepresent invention is not limited thereto.

Also, the controller 1100 may generate a first property data 3201 and asecond property data 3202 on the basis of the first subset of point data3110 and the second subset of point data 3120 included in the first setof point data received from the infrastructure device 700. In this case,the first property data 3201 may include class information, centerposition information, size information, etc. of the second vehicle 123which are represented by the first subset of point data 3110, but thepresent invention is not limited thereto. Also, the second property data3202 may include class information, center position information, sizeinformation, etc. of the pedestrian 800 which are represented by thesecond subset of point data 3120, but the present invention is notlimited thereto.

FIG. 38 is a diagram illustrating a method of processing, by a firstvehicle, a shared set of point data and a second set of point dataaccording to another embodiment.

Referring to FIG. 38 , the controller of the first vehicle 122 maygenerate a third set of point data 4100 using the second set of pointdata 2100 and the shared first set of point data 3100 to recognize atleast one object included in the field of view of the sensor.

In this case, the third set of point data 4100 may be generated byaligning the coordinate system of the shared first set of point data3100 with the coordinate system of the second set of point data 2100.The coordinate system alignment will be described in detail below (inSection 3.4.2.).

Also, the third set of point data 4100 may include a fourth subset ofpoint data 4110 representing the second vehicle 123 and a fifth subsetof point data 4120 representing the pedestrian 800.

Also, the controller 1100 may generate fourth property data 4201 on thebasis of the fourth subset of point data 4110 and may generate fifthproperty data 4202 on the basis of the first subset of point data 4120.In this case, the fourth property data 4201 may include classinformation, center position information, size information, etc. of thesecond vehicle 123 which are represented by the fourth subset of pointdata 4110, but the present invention is not limited thereto. Also, thefifth property data 4202 may include class information, center positioninformation, size information, etc. of the pedestrian 800 which arerepresented by the fifth subset of point data 4120, but the presentinvention is not limited thereto.

3.4.1.2. Case of Receiving Property Data

FIG. 39 is a diagram illustrating a situation in which property data isincluded in the content of sharing data according to an embodiment.

Referring to FIG. 39 , the infrastructure device 700 may transmit, tothe first vehicle 122, sharing data 3000 including a plurality of piecesof property data 3200 generated based on a plurality of subsets of pointdata included in a set of point data acquired from a sensor.

When the sharing data 3000 is received, the controller 1100 of the firstvehicle 122 may control the first vehicle 122 using the sharing data3000.

A method in which the first vehicle 122 that has received the sharingdata 3000 including the plurality of property data 3200 processes thesharing data 3000 will be described in detail below (in Section 5.).

3.4.1.3. Case of Receiving Event Occurrence-Related Information

Referring to FIG. 35 again, the server 400, the vehicles 122 and 123,and the infrastructure device 700, each of which includes acommunication module, may share sharing data 3000 including eventoccurrence-related information.

For example, the server 400 may transmit event-related informationincluding information indicating that a traffic event has occurred onthe path of the first vehicle 122 to the first vehicle 122.

A method in which the first vehicle 122 that has received the sharingdata 3000 including the event occurrence-related information processesthe sharing data 3000 will be described in detail below (in Section4.2.).

3.4.2. Coordinate System Alignment for Shared-Data Matching

When a first device receives sharing data from a second device, acontroller 1100 of the first device may match the coordinate system ofsensor data acquired from a sensor placed in the first device to thecoordinate system of the sharing data in order to match the sensor datato the sharing data (data registration).

In this case, the coordinate system may include a Cartesian coordinatesystem, a polar coordinate system, a cylindrical coordinate system, ahomogeneous coordinate system, a curved coordinate system, an inclinedcoordinate system, a log-polar coordinate system, or the like, but thepresent invention is not limited thereto.

For example, a first device including a first LiDAR device may acquirefirst sensor data through the first LiDAR device. Also, a second deviceincluding a second LiDAR device may acquire second sensor data throughthe second LiDAR device. In this case, the first LiDAR device mayinclude a first local coordinate system having a first LiDAR-opticalorigin as the origin. Also, the second LiDAR device may include a secondlocal coordinate system having a second LiDAR-optical origin as theorigin.

Here, when a controller of the second device transmits sharing dataincluding the second sensor data to the first device, the controller ofthe first device may set the first local coordinate system as a globalcoordinate system. Also, after receiving the shared second sensor datashown in the second local coordinate system, the controller may alignthe second local coordinate system with the global coordinate system inorder to perform matching on the second sensor data. Also, in someembodiments, the controller may align the second local coordinate systemwith the first local coordinate system or align the first localcoordinate system with the second local coordinate system. It will beappreciated that the first local coordinate system is the same as thesecond local coordinate system.

Also, in order to align the second local coordinate system with theglobal coordinate system in a 3D space, the controller may compute a 4×4transformation matrix with a total of six degrees of freedom (6DOF) bysumming a 3D vector for translation and a 3D vector for rotation. Also,the controller may transform the second sensor data shown in the secondlocal coordinate system to the global coordinate system using thetransformation matrix.

As an example, when the first device is fixed, the alignment between thesecond local coordinate system and the local coordinate system may beperformed by computing a transformation relationship between thecoordinate systems. That is, the controller may transform the sensordata shown in the second coordinate system into the global coordinatesystem using the transformation matrix to show the sensor data in aunified communication system.

As another example, in order to align the second local coordinate systemwith the local coordinate system in a 3D space, the controller may use afirst object having a unique shape as a criterion for the alignment. Forexample, the unique shape may include a shape in which three planes meetin the first object, but the present invention is not limited thereto.In detail, the controller may align the second local coordinate systemwith the local coordinate system on the basis of the position of a firstobject included in second sensor data shown in the second localcoordinate system and the position of the unique shape of a first objectincluded in first sensor data shown in the global coordinate system.

Specifically, the controller may generate an initial position bymatching the position of the first object shown in the global coordinatesystem and the position of the first object shown in the second localcoordinate system. In this case, the initial position may be acquired byinitially aligning the positions of the unique shape of the first objectincluded in different pieces of sensor data with the global coordinatesystem. That is, the initial position alignment process may beunderstood as the initial coordinate system alignment. Also, whenposition information (e.g., an initial portion) of the first objectacquired from different devices shown in the local coordinate system isincorrect, the controller can improve the position information of thefirst object through optimization. In this case, the controller may usean iterative closest point (ICP) algorithm to optimize the initialposition, but the present invention is not limited thereto.

3.5. Vehicle Control Using Sharing Data

A controller included in a vehicle that has received sharing data maycontrol the vehicle using the sharing data and sensor data acquired froma sensor of the vehicle. In this case, it will be appreciated that theembodiment of vehicle control using sensor data, which has beendescribed in Sections 2.3.1 to 2.3.3, can also be implemented usingsharing data.

In detail, the controller may match the sharing data, which is receivedfrom another device, to a high-definition map included in the vehicleand display the matching result.

Also, the controller may control the direction and speed of the vehicleusing the sharing data received from another device.

Also, the controller may control the path of the vehicle using thesharing data received from another device.

4. Selective Sharing of Sensor Data 4.1. Selective Sharing of SensorData According to Property Data

A data sharing system according to an embodiment may include a firstdevice and a second device. Also, the first device may transmit sharingdata to the second device.

In this case, the content of the sharing data transmitted by the firstdevice may differ depending on an object recognition result included insensor data acquired by the first device. Here, the object recognitionresult may refer to a class information of the object.

For example, when the class of the object included in the classinformation is related to a building, the content of the sharing datamay include a subset of point data representing the building. Also, whenthe class of the object included in the class information is a class inwhich personal information needs to be protected, the content of thesharing data may include property data of a subset of point datarepresenting the object.

Here, the class in which personal information needs to be protectedrefers to a class in which personal information may be exposed, such asa human, a vehicle number plate, and an identity document and the classin which personal information needs to be protected may be predeterminedby the controller.

That is, the controller of the first device may selectively generatesharing data according to the class information of the object includedin the sensor data.

4.1.1. Necessity of Selective Sharing of Sensor Data According toProperty Data

In the data sharing system according to an embodiment, privacy may beunjustly invaded when data related personal information is randomlyshared between a plurality of devices. For example, when a photoincluding a person's face is transmitted to another device without anyprocessing, his or her privacy may be invaded when shape and colorinformation related to his or her face is shared.

Also, even when the sensor is a LiDAR device, privacy invasion can be anissue. In detail, sensor data acquired from the LiDAR device may includeintensity information of an object. Here, since the intensityinformation includes an intensity value that is different depending onthe reflectance of the object, a controller connected to the LiDARdevice may determine a human face using the intensity information. Thus,even when sensor data acquired from the LiDAR device is shared between aplurality of devices without being processed, privacy invasion can be anissue. Thus, a method of selectively sharing sensor data according to anobject class may be required when the sensor data is shared.

In a data sharing system according to another embodiment, a deviceincluding at least one sensor may selectively share sensor data in orderfor a device for generating a high-definition map to efficiently updatethe high-definition map. In an embodiment, a high-definition map that isinitially generated may require sensor data for movable objects such aspeople rather than sensor data for immovable objects such as buildings.Accordingly, a device for transmitting the sensor data may select onlydata related to immovable objects from the sensor data and transmit thedata to the device for generating the high-definition map.

In the data sharing system according to still another embodiment,information on immovable objects may be prestored in a high-definitionmap. In this case, the device for transmitting the sensor data mayselect only data related to movable objects from the sensor data andtransmit the data to the device for generating the high-definition map.In this case, the device for generating the high-definition map maygenerate a high-definition map including both of an immovable object anda movable object by additionally acquiring data related to the mobileobject in addition to the prestored information on the immovableobjects.

4.1.2. Various Embodiments of Selective Sharing Method of Sharing DataIncluding Privacy Protection Data

In order to solve the above-described privacy invasion issue, thesharing data may include privacy protection data. Here, the privacyprotection data may be data obtained by processing a personalinformation identification-related part in a plurality of subsets ofpoint data included in a set of point data. The privacy protection datawill be described in detail below (in Section 4.1.2.1.3.).

4.1.2.1. Selective Sharing Method According to Embodiment

A data sharing system according to an embodiment may include a firstdevice and a second device, each of which includes at least onecommunication module for performing communication. In this case, thefirst device and the second device may include a vehicle, a server, aninfrastructure device, a mobile device, or the like, but the presentinvention is not limited thereto.

FIG. 40 is a flowchart illustrating a selective sharing method of sensordata according to an embodiment.

Referring to FIG. 40 , a controller of a first device may obtain a setof point data 2100 through at least one sensor (S5001). In this case,the set of point data 2100 may correspond to a point cloud acquiredthrough a LiDAR device. Also, the first device may include a vehicle, aninfrastructure device, a server, a mobile device, etc., but the presentinvention is not limited thereto.

Also, the controller may determine property data of a plurality ofsubsets of point data included in the set of point data (S5002).

Also, the controller may determine class information of an each objectrepresented by each of the plurality of subsets of point data (S5003).

Also, the controller may change the content of the sharing dataaccording to whether the class of the object included in the classinformation is a class in which personal information needs to beprotected (S5004).

Also, the controller may generate sharing data including privacyprotection data when the class of the object included in the classinformation is a class in which personal information needs to beprotected (S5005) and may generate sharing data not including privacyprotection data when the class of the object included in the classinformation is not a class in which personal information needs to beprotected (S5006).

Also, the controller may transmit the generated sharing data to a seconddevice (S5007).

The operations described in FIG. 40 will be described in detail below onthe assumption that the first device is a first vehicle 124.

4.1.2.1.1. Acquisition of Sensor Data

Referring to FIG. 40 again, a controller of the first vehicle 124 mayobtain a set of point data through at least one sensor (S5001). In thiscase, the set of point data may include a plurality of pieces of pointdata. Also, the set of point data may include a plurality of subsets ofpoint data representing at least a portion of an object. Also, the atleast one sensor may include a LiDAR device, a camera device, a radardevice, an ultrasonic sensor, or the like, but the present invention isnot limited thereto.

FIG. 41 is a diagram showing a situation in which a first vehicleacquires sensor data to selectively share the sensor data according toan embodiment.

FIG. 42 is a diagram schematically representing the sensor data acquiredby a first vehicle through a LiDAR device in FIG. 41 in a 2D plane.

Referring to FIGS. 41 and 42 , the controller of the first vehicle 124may acquire a set of point data 2101 including a plurality of subsets ofpoint data 2111 and 2112 through at least one sensor.

In this case, the controller of the first vehicle 124 may extract theplurality of subsets of point data 2111 and 2112 included in the set ofpoint data 2101 and may determine property data including classinformation of the plurality of subsets of point data 2111 and 2112(S5002, S5003).

In detail, the controller may extract a first subset of point data 2111representing at least a portion of a third vehicle 126 and a secondsubset of point data 2112 representing at least a portion of apedestrian 800 from the set of point data 2101.

Also, the controller may acquire the first subset of point data 2111 andthe second subset of point data 2112 in the scheme described in Section2.2.3.

FIG. 43 is a diagram showing class information and property data of aplurality of subsets of point data included in sensor data according toan embodiment.

Referring to FIG. 43 , the controller may determine a plurality ofpieces of property data 2201 and 2202 corresponding to the plurality ofsubsets of point data 2111 and 2112 on the basis of the plurality ofsubsets of point data 2111 and 2112, respectively.

More specifically, the controller may determine first property data 2201including first class information 2211 on the basis of the first subsetof point data 2111. In this case, the first class information 2211 mayrepresent “vehicle.” However, the present invention is not limitedthereto, and the first class information 2211 may be determined as“passenger car,” which is a subclass of “vehicle.”

Also, the controller may determine second property data 2202 includingsecond class information 2212 on the basis of the second subset of pointdata 2120. In this case, the second class information 2212 may represent“human.” However, the present invention is not limited thereto, and thesecond class information 2212 may be determined as “pedestrian,” whichis a subclass of “human.”

Also, the controller may acquire a plurality of pieces of property data2201 and 2202 including a plurality of pieces of class information 2211and 2212 in the scheme described in Section 2.2.4.

4.1.2.1.2. Selective Generation and Sharing of Sharing Data

Also, the controller may generate sharing data 3000 in order to transmitthe sensor data 2000 to a second device.

In this case, in order not to share data related to privacy, a criterionfor determining the content of the sharing data may be required. Forexample, the sharing data 3000 may be generated differently depending onclass information of a plurality of subsets of point data 2111 and 2112included in the sensor data 2000. Here, the controller may determine thecontent of the sharing data according to whether the class informationis related to personal information identification. However, the presentinvention is not limited thereto, and the controller may determine thecontent of the sharing data on the basis of the plurality of pieces ofproperty data 2201 and 2202.

FIG. 44 is a diagram showing the content of sharing data transmitted bya first vehicle according to an embodiment.

Referring to FIG. 44 , the controller of the first vehicle 124 maydetermine the content of sharing data on the basis of class informationof a plurality of objects included in the set of point data 2101.

Also, the controller may determine the content of the sharing dataaccording to whether the property data is related to personalinformation identification. In detail, the controller may determine thecontent of the sharing data according to whether the class of an objectincluded in the class information is a class in which personalinformation needs to be protected.

As an example, the controller may determine the content of sharing dataaccording to whether the class information is related to a human. Inthis case, the controller may generate sharing data that does notinclude at least one piece of point data representing a human face.Also, the controller may generate sharing data including data obtainedby processing the at least one piece of point data representing thehuman face.

As another example, the controller may not add data related to a vehiclenumber plate among sensor data related to a vehicle to the content ofthe sharing data. Also, the controller may generate sharing dataincluding data obtained by processing at least one piece of point datarepresenting the number plate of the vehicle.

Also, the controller may determine the content of the sharing dataaccording to whether the class information of the object matches atleast one class included in a class group related to personalinformation. In this case, the class group may be a collection ofclasses including at least one class that satisfies a preset criterion.For example, the class group related to personal information may includea class related to a human, a class related to a number plate, a classrelated to an identity document, or the like, but the present inventionis not limited thereto.

For example, when class information of an object acquired through atleast one sensor is determined as “human,” the controller may not add asubset of point data representing at least a portion of the object tothe content of the sharing data for sharing information on the object.However, the present invention is not limited thereto, and thecontroller may generate shard data including data obtained by processinga part related to a human face in the subset of point data.

Also, the first vehicle 124 may transmit sharing data to the seconddevice (S5007). In this case, the second device may include vehicles 125and 126, a server 400, an infrastructure device 700, a mobile device,etc., but the present invention is not limited thereto.

For example, referring to FIG. 44 again, when the second device is asecond vehicle 125, the first vehicle 124 may transmit the sharing data3000 to the second vehicle 125. In this case, the content of the sharingdata may include the privacy protection data 3300, the first subset ofpoint data 2111, etc., but the present invention is not limited thereto.

In this case, the content of the sharing data may be determined based onclass information of the plurality of subsets of point data 2111 and2112. In detail, since the class information 2211 of the first subset ofpoint data representing at least a portion of the third vehicle 126 isrelated to a vehicle, the content of the sharing data may include thefirst subset of point data 2111. However, the present invention is notlimited thereto. Since the number plate of the vehicle may be related topersonal information identification, the content of the sharing data mayinclude privacy protection data obtained by processing at least onepiece of point data representing the number plate of the vehicle.

Also, since class information of the second subset of point datarepresenting at least a portion of the pedestrian 800 is related to ahuman, which is a class in which personal information needs to beprotected, the content of the sharing data may include the privacyprotection data 3300.

4.1.2.1.3. Privacy Protection Data

Also, when a class included in class information of at least one subsetof point data included in the set of point data 2101 is a class in whichpersonal information needs to be protected, the controller may generatesharing data 3000 including privacy protection data 3300 (S5005).

In this case, the controller may generate the privacy protection data3300 in order not to share data related to personal informationidentification. In other words, the privacy protection data 3300 may begenerated to protect privacy.

Also, the privacy protection data 3300 may not include data related topersonal information identification. In detail, since the subset ofpoint data includes intensity information of an object, the subset ofpoint data may be data related to personal information identification.Thus, the privacy protection data 3300 may not include a personalinformation identification-related part of the subset of point data.Also, the privacy protection data 3300 may include property data of thesubset of point data. Also, the privacy protection data 3300 may includedata obtained by processing the personal informationidentification-related part of the subset of point data.

FIG. 45 is a diagram illustrating privacy protection data included inthe content of sharing data according to an embodiment.

Referring to FIG. 45 , the privacy protection data 3300 may include thesecond property data 2202 generated based on the second subset of pointdata 2112.

For example, the privacy protection data 3300 may include centerposition information 2221 representing the center position of thepedestrian 800. In detail, the controller may generate privacyprotection data 3300 including the center position informationrepresenting the center coordinates of a plurality of pieces of pointdata included in the second subset of point data 2112.

Also, the privacy protection data 3300 may include size information 2231representing the size of the pedestrian 800. In detail, the controllermay generate privacy protection data 3300 including the size information2231 representing a volume value of the pedestrian 800 represented bythe second subset of point data 2112.

Also, the privacy protection data 3300 may include shape information2240 represented by processing the shape of the pedestrian 800. Indetail, the controller may generate privacy protection data 3300 inwhich the second subset of point data 2112 is replaced withpredetermined template information 2241 according to the classinformation of the second subset of point data 2112. Also, thecontroller may generate privacy protection data 3300 including skeletoninformation 2242 representing the second subset of point data 2112 as atleast one point.

However, the present invention is not limited thereto, and the privacyprotection data 3300 may include at least some of a plurality of piecesof information included in the second property data. For example, theprivacy protection data 3300 may include at least some of centerposition information, size information, movement information, shapeinformation, identification information, and class information of thesecond subset of point data, but the present invention is not limitedthereto.

Referring to FIG. 45 again, the privacy protection data 3300 may includedata 3310 obtained by processing at least a portion of the second subsetof point data 2112.

For example, the privacy protection data 3300 may include data obtainedby pixelating at least some of the plurality of pieces of point dataincluded in the second subset of point data 2112. In detail, thecontroller may generate privacy protection data 3300 obtained bypixelating at least one piece of point data related to the face of thepedestrian in the second subset of point data 2112 representing at leasta portion of the pedestrian 800.

Also, the privacy protection data 3300 may include data obtained byblurring out at least a portion of the second subset of point data 2120.In detail, the controller may generate privacy protection data 3300obtained by blurring out at least one piece of point data related to theface of the pedestrian 800 in the second subset of point data 2112representing at least a portion of the pedestrian 800.

Also, the privacy protection data 3300 may include data obtained byadding noise data to at least a portion of the second subset of pointdata 2112. In detail, the controller may generate privacy protectiondata 3300 obtained by adding the noise data to a part related to theface of the pedestrian 800 in the second subset of point data 2112representing at least a portion of the pedestrian 800.

Also, the privacy protection data 3300 may include data obtained byremoving at least a portion of the second subset of point data 2112. Indetail, the controller may generate privacy protection data 3300obtained by removing at least some of the plurality of pieces of pointdata related to the face of the pedestrian 800 from the second subset ofpoint data 2112 representing at least a portion of the pedestrian 800.

Also, the privacy protection data 3300 may include data obtained byremoving a subset of point data representing an object with a class inwhich personal information needs to be protected. For example, thecontroller may generate privacy protection data 3300 obtained byremoving the second subset of point data 2112 representing at least aportion of the pedestrian 800.

Also, the privacy protection data 3300 may include data obtained bydeleting intensity information of at least a portion of the secondsubset of point data 2112. In detail, the controller may generateprivacy protection data 3300 in which intensity values of a plurality ofpieces of point data related to a human face in the second subset ofpoint data 2112 are set to zero.

Also, when the sensor is a camera device, the privacy protection data3300 may include data in which a pixel value of the camera device is setto any value. For example, the controller may generate privacyprotection data 3300 in which a pixel value of a part representing theface of the pedestrian 800 in the second subset of point data 2112 isadjusted to any value.

However, the present invention is not limited thereto, and the privacyprotection data 3300 may include data obtained by processing at least aportion of the second subset of point data 2112 using a predetermineddata processing technique. The predetermined data processing techniquecan be used by those skilled in the art, and thus a detailed descriptionthereof will be omitted.

4.1.2.2. Selective Sharing According to Other Embodiment

A data sharing system according to another embodiment may requireapproval from a server placed an external institution beforetransmitting sharing data. For example, the data sharing system mayrequire approval for sharing sensor data itself from an externalinstitution or may require approval for sharing data related to personalinformation identification included in sensor data from an externalinstitution. In this case, the external institution may include agovernment institution, a data management institution, etc. However, thepresent invention is not limited thereto, and the external institutionmay perform communication through a server.

FIG. 46 is a flowchart illustrating a method of selectively sharing datadepending on whether approval for data sharing is gained from anexternal server in a data sharing system according to an embodiment.

Referring to FIG. 46 , a controller of a first device may acquire a setof point data 2100 through at least one sensor (S5008).

Also, the controller may determine property data of a plurality ofsubsets of point data included in the set of point data (S5009).

Also, the controller may determine class information of an objectrepresented by each of the plurality of subsets of point data (S5010).

Also, the controller may determine whether approval for transmitting theplurality of subsets of point data to another device is gained from anexternal server (S5011). In this case, the external server may determinewhether there is a need to share the plurality of subsets of point datadespite a privacy invasion issue that may arise by transmitting theplurality of subsets of point data. For example, when at least one ofthe plurality of subsets of point data represents at least a portion ofa criminal involved in at least one crime situation, the external servermay approve the sharing of a subset of point data representing at leasta portion of the criminal.

Also, the controller may request approval for transmitting the sharingdata from the external server. In this case, the controller may requestthe approval while transmitting a subset of point data related topersonal information identification to the external server. However, thepresent invention is not limited thereto, and the controller may requestthe approval while transmitting property data (e.g., class information)of the subset of point data to the external server. Also, when theapproval request is received, the external server may determine whetherto approve the transmission of the sharing data.

Also, even when there is no approval request from the controller, theexternal server may determine whether to approve of the controllertransmitting the sharing data.

Also, once the external server approves the transmission of the sharingdata, the approval from the external server is no longer needed to sharedata related to an object represented by a subset of point data includedin the content of sharing data. However, the present invention is notlimited thereto, and the controller may gain approval from the externalserver each time the sharing data is transmitted.

Also, when there is approval from the external server, the controllermay generate sharing data including the plurality of subsets of pointdata regardless of the class information of the plurality of subsets ofpoint data (S5013). For example, even when a subset of point datarepresenting at least a portion of a human is included in the pluralityof subsets of point data, the controller may generate sharing dataincluding a subset of point data representing at least a portion of thehuman without generating privacy protection data.

Also, when there is no approval from the external server, the controllermay determine whether the class of an object included in the classinformation is a class in which personal information needs to beprotected (S5012).

Also, when the class information is related to personal informationidentification, the controller may generate sharing data includingprivacy protection data (S5013).

Also, when the class information is not related to personal informationidentification, the controller may generate sharing data including noprivacy protection data (S5014). Here, the content of the sharing datamay include a subset of point data.

Also, the controller may transmit the sharing data to a second device(S5015). In this case, the second device may include a vehicle, aserver, an infrastructure device, a mobile device, etc., but the presentinvention is not limited thereto.

4.1.2.3. Whether to Generate Privacy Protection Data According toPosition of Sensor

Whether to generate privacy protection data according to an embodimentmay be determined depending on the position of at least one sensor thatacquires sensor data. For example, the at least one sensor may be placedin a vehicle, but the present invention is not limited thereto.

As a specific example, for a vehicle including an autonomous drivingsystem according to an embodiment, at least one sensor 1300 included inthe autonomous driving system 1000 may be placed in the vehicle. In thiscase, the at least one sensor 1300 may acquire sensor data includingposition information and shape and/or color information of an occupantof the vehicle.

In this case, a controller of the autonomous driving system may generateprivacy protection data regardless of class information of an objectincluded in the sensor data. In detail, when the vehicle is not anunmanned vehicle, it is essential that an occupant gets in the vehicle,and thus the controller may always generate privacy protection data onthe basis of the sensor data.

Also, the controller may generate privacy protection data according towhether a subset of point data representing at least a portion of ahuman is included in the sensor data. In this case, the controller maydetermine whether a subset of point data with a class related to a humanis included in the sensor data by determining class information of thesubset of point data as described above. Also, the controller mayacquire information regarding whether an occupant is in the vehicle fromany device placed in the vehicle. For example, the controller maydetermine whether an occupant is in the vehicle by acquiring vehicleriding information through a weight detection sensor placed in thevehicle.

Also, the controller 1100 of the vehicle may generate sharing data 3000for transmitting the sensor data 2000 to another device through at leastone communication module 1200. In this case, the content of the sharingdata may include privacy protection data 3300. In detail, the controller1100 may generate privacy protection data 3300 for personal informationprotection regardless of the class information of an object included inthe sensor data.

4.1.2.4. Whether to Generate Privacy Protection Data According toDistance and Intensity Information

At least one sensor included in an autonomous driving system using adata sharing system according to an embodiment may include a LiDARdevice. In this case, the LiDAR device may acquire intensity informationaccording to the reflectance and distance information of an objectlocated within a field of view. In this case, a controller included inthe autonomous driving system may determine whether to generate privacyprotection data according to the distance information and the intensityinformation.

As an example, when an object is spaced a certain distance from theLiDAR device, the controller cannot identify personal information of theobject on the basis of sensor data acquired from the LiDAR device. Inthis case, when a distance between a first device including thecontroller and a first object included in the sensor data is greaterthan or equal to a predetermined distance, the controller may notgenerate privacy protection data regardless of the class of the firstobject. The predetermined distance may refer to a distance at thepersonal information of the first object is not identified through thesubset of point data regardless of the reflectance of the first object.Also, the controller may preset and store the predetermined distance orset the predetermined distance on the basis of sensor data.

As another example, when the reflectance of an object is low, thecontroller cannot identify personal information of the object throughthe LiDAR device. In this case, when an intensity value of a secondobject is less than or equal to a threshold, the controller may notgenerate privacy protection data regardless of the class of the secondobject. In this case, the threshold may refer to an intensity value inwhich the personal information of the second object is not identifiedthrough the subset of point data regardless of distance information ofthe second object. Also, the controller may preset and store thethreshold and set the threshold on the basis of sensor data.

Also, the controller may generate sharing data including at least one ofa plurality of subsets of point data representing at least a portion ofthe first object or the second object and property data of the pluralityof subsets of point data.

4.1.2.5. Selective Storing of Sensor Data for Privacy Protection

The embodiments of selectively storing sensor data to protect privacymay be applied to a case of selectively storing the sensor data.

For example, when the class of an object included in the classinformation of the subset of point data is a class in which personalinformation needs to be protected, a device that acquires the subset ofpoint data may not store the subset of point data. In this case, thedevice may generate and store privacy protection data obtained byprocessing at least a portion of the sub set of point data.

However, the present invention is not limited thereto, and the devicemay always store the subset of point data regardless of the classinformation of the subset of point data.

4.1.3. Selective Sharing of Sharing Data to Generate High-Definition Map4.1.3.1. Selective Sharing Method According to Embodiment

A data sharing system according to an embodiment may include a firstdevice and a second device, each of which includes at least onecommunication module for performing communication. In this case, thefirst device and the second device may include a vehicle, a server, aninfrastructure device, a mobile device, or the like, but the presentinvention is not limited thereto.

FIG. 47 is a flowchart illustrating a detailed method of selectivelysharing sensor data according to another embodiment.

Referring to FIG. 47 , a controller of a first device may obtain a setof point data through at least one sensor (S5017).

Also, the controller may determine class information of a subset ofpoint data included in the set of point data (S5018).

Also, the controller may determine whether an object represented by thesubset of point data is movable on the basis of the class information(S5019).

Also, when the object cannot move, the controller may generate sharingdata including the subset of point data (S5020).

Also, the controller may transmit the sharing data to a second device(S5021).

Hereinafter, each operation will be described in detail.

4.1.3.1.1. Acquisition of Sensor Data

Referring to FIG. 47 again, a controller of a first device may obtain aset of point data through at least one sensor (S5017). Also, thecontroller may determine class information of a plurality of subsets ofpoint data included in the set of point data (S5018). In this case, thefirst device may include a vehicle, an infrastructure device, etc., butthe present invention is not limited thereto.

FIG. 48 is a diagram showing a situation in which a first vehicleacquires sensor data to selectively share the sensor data according toan embodiment.

FIG. 49 is a diagram schematically representing sensor data acquired bythe first vehicle through a LiDAR device according to FIG. 48 in a 2Dplane.

Referring to FIGS. 48 and 49 , a controller of a first vehicle 127 mayobtain a set of point data 2102 including a plurality of subsets ofpoint data 2113, 2114, and 2115 through at least one sensor.

For example, the controller may extract a first subset of point data2113 representing at least a portion of a pedestrian 800, a secondsubset of point data 2114 representing at least a portion of a thirdvehicle 129, and a third subset of point data 2115 representing at leasta portion of a building 500 in the set of point data.

Also, the controller may determine class information of the plurality ofsubset of point data 2113, 2114, and 2115. For example, the controllermay determine that the class information of the first subset of pointdata 2113 is “human.” However, the present invention is not limitedthereto, and the controller may determine that the class information isa sub class of “human.” Also, the controller may determine that theclass information of the second subset of point data 2114 is “vehicle.”However, the present invention is not limited thereto, and thecontroller may determine that the class information is a sub class of“vehicle.” Also, the controller may determine that the class informationof the third subset of point data 2115 is “building.” However, thepresent invention is not limited thereto, and the controller maydetermine the class information as a sub class of “building.”

4.1.3.1.2. Criterion for Selecting Sharing Data

Also, the controller may determine whether an object represented by thesubset of point data is movable on the basis of the class information(S5019).

In detail, in order to selectively share sensor data according to anembodiment, the controller may determine the movability of objectsrepresented by the plurality of subsets of point data 2113, 2114, and2115.

In this case, whether the objects are movable may be determined based onclass information of the objects.

More specifically, referring to FIG. 49 , the controller may determinethat a pedestrian 800 and a third vehicle 129 are movable objects on thebasis of class information of the first subset of point data 2113representing at least a portion of the pedestrian 800 and the secondsubset of point data 2114 representing at least a portion of the vehicle129. Also, the controller may determine that a building 500 is animmovable object on the basis of class information of the third subsetof point data 2115 representing at least a portion of the building 500.

As an example, the controller may determine the movability of an objecton the basis of whether class information of a subset of point datarepresenting the object is related to an immovable object or is relatedto a movable object. For example, when the controller determines thatthe class information of the third subset of point data 2115 is“building,” the class information is related to an immovable object.Thus, the controller may determine that the building 500 represented bythe third subset of point data 2115 is immovable.

As another example, the controller may pre-classify class informationinto a movable object and an immovable object and may determine thatclass information of a subset of point data representing the object is amovable object or an immovable object. For example, the controller maydetermine that the class information of the third subset of point data2115 is “immovable object.” In this case, the controller may determinethat the building 500 represented by the third subset of point data 2115is immovable.

Also, the controller may determine the content of sharing data accordingto a class type of an object on the basis of class information of asubset of point data without determining the movability of the object onthe basis of the class information of the subset of point data.

In detail, the controller may determine the content of the sharing dataaccording to a predetermined criterion on the basis of the class type ofthe object included in the class information of the subset of pointdata. That is, a predetermined criterion for determining the content ofthe sharing data may be predetermined for each class type of the object.

As an example, the content of the sharing data may not include the firstsubset of point data when the class type of the object included in theclass information of the first subset of point data is “human” or“vehicle” and may include the second subset of point data when the classtype of the object included in the class information of the secondsubset of point data is a class other than “human” or “vehicle.”

As another example, the content of the sharing data may include thefirst subset of point data when the class type of the object included inthe class information of the first subset of point data is an immovableobject such as “building” and may not include the second subset of pointdata when the class type of the object included in the class informationof the second subset of point data is a class other than an immovableobject such as “building.”

It will be appreciated that the predetermined criterion for the classtype may vary depending on the embodiment. For example, the content ofthe sharing data may be determined according to a criterion contrary tothe above-described predetermined criterion, but the present inventionis not limited thereto.

Also, a user may set the predetermined criterion while designing thedata sharing system according to an embodiment and may also use thepredetermined criterion while using the data sharing system.

4.1.3.1.3. Generation and Transmission of Sharing Data

Also, when the object is immovable, the controller may generate sharingdata including the subset of point data (S5020).

In detail, in order to selectively share sensor data according to anembodiment, the controller may generate sharing data on the basis of themovability of a plurality of objects represented by the plurality ofsubsets of point data 2113, 2114, and 2115.

As an example, when class information of a subset of point data isrelated to an immovable object, the controller may generate sharing dataincluding at least a portion of the subset of point data or the propertydata of the subset of point data.

FIG. 50 is a diagram illustrating the content of sharing data accordingto an embodiment.

Referring to FIG. 50 , the content of the sharing data 3000 may includea third subset of point data 2115 having class information related to animmovable object.

In this case, since the third subset of point data 2115 represents atleast a portion of the building 500, which is an immovable object, thecontroller may generate the sharing data 3000 including the third subsetof point data 2115.

However, the present invention is not limited thereto, and when theclass information of the third subset of point data 2115 is related toan immovable object, the controller may generate sharing data 3000including third property data 2205 of the third subset of point data.

In this case, the third property data 2205 may include at least some ofclass information, center position information, size information, shapeinformation, movement information, or identification information whichis acquired based on the third subset of point data 2115.

However, the present invention is not limited thereto, and even whenclass information of a subset of point data is related to a movableobject, the controller may generate sharing data including property dataof the subset of point data. In detail, when the class information ofthe subset of point data is related to a movable object, the controllermay generate sharing data including center position information of thesubset of point data.

For example, the content of the sharing data may further include firstand second property data 2203 and 2204 of the first and second subsetsof point data 2113 and 2114 having class information related to themovable object.

In this case, the first and second property data 2203 and 2204 mayinclude center position information acquired based on the first andsecond subsets of point data 2113 and 2114, but the present invention isnot limited thereto.

Also, the first vehicle 127 may transmit the sharing data to a seconddevice.

In this case, the second device may include vehicles 128 and 129, aninfrastructure device 700, a server 400, a mobile device, etc., but thepresent invention is not limited thereto.

For example, when the second device is a server 400, the first vehicle127 may transmit the sharing data 3000 to the server 400. In this case,the server 400 may generate a high-definition map on the basis of thesharing data.

4.1.3.2. Sharing Data Including Additional Information

Also, the content of the sharing data may include additional informationrelated to a stop time of a stationary object.

In detail, when class information of an object included in the sensordata is related to a stationary object and additional informationrelated to a stop time of the object is included in the sensor data, thecontroller may generate sharing data including the additionalinformation.

FIG. 51 is a flowchart illustrating a method of selectively sharingsensor data including additional information according to an embodiment.

Referring to FIG. 51 , a controller of a first device may obtain a setof point data through at least one sensor and determine classinformation of a plurality of subsets of point data included in the setof point data (S5022). In this case, the first device may include avehicle, an infrastructure device, etc., but the present invention isnot limited thereto. Also, the class information may be related to astationary object or may be related to a movable object.

Also, the controller may determine the movability of a plurality ofobjects represented by the plurality of subsets of point data on thebasis of the class information (S5023). For example, when an object isrelated to a stationary object, the controller may determine that theobject is immovable.

Also, for an object determined to be immovable, the controller mayobtain additional information related to movability (S5024). In thiscase, the additional information may include a stop time of thestationary object.

Also, when the additional information is acquired, the controller maygenerate sharing data including the additional information and thesubset of point data (S5025).

Also, when the additional information is not acquired, the controllermay generate sharing data including the subset of point data (S5026).

Also, the controller may transmit the sharing data to a second device(S5027). In this case, the second device may include a vehicle, aninfrastructure device, a server, etc., but the present invention is notlimited thereto.

FIG. 52 is a diagram showing a situation in which a first vehicleacquires additional information through at least one sensor according toan embodiment.

FIG. 53 is a diagram schematically showing, in a 2D plane, the sensordata acquired by the first vehicle according to FIG. 52 .

Referring to FIGS. 52 and 53 , a first vehicle 130 may acquire a set ofpoint data 2103 including a plurality of subsets of point data 2116,2117, and 2118 through at least one sensor.

In this case, the plurality of subsets of point data 2116, 2117, and2118 may include a first subset of point data 2116 representing at leasta portion of a construction sign 900, a second subset of point data 2117representing at least a portion of a third vehicle 132, and a thirdsubset of point data 2118 representing at least a portion of a building500, but the present invention is not limited thereto.

Also, a controller of the first vehicle may determine class informationof the plurality of subsets of point data. For example, the controllermay determine that the class information of the first subset of pointdata 2116 is “sign,” determine that the class information of the secondsubset of point data 2117 is “vehicle,” and determine that the classinformation of the third subset of point data 2118 is “building.”

Also, the controller may determine whether class information of aplurality of objects is related to an immovable object to determine themovability of the plurality of objects. For example, since the classinformation of the first subset of point data 2116 and the third subsetof point data 2118 is related to an immovable object, the controller maydetermine that the construction sign 900 and the building are immovable.

Also, the controller may generate sharing data including a subset ofpoint data representing an object that cannot move. In detail, whenadditional information related to a stop time of an object is includedin the subset of point data representing the immovable object, thecontroller may generate sharing data further including the additionalinformation. For example, the controller may add additional informationrelated to the stop time of the construction sign (e.g., informationregarding a construction period) to the first subset of point data 2116.In this case, the additional information may be acquired based onintensity information of the construction sign 900 acquired from atleast one LiDAR device. In detail, the controller may recognizeadditional information representing a construction completion time shownin the construction sign 900 on the basis of an intensity value includedin the first subset of point data 2116 representing at least a portionof the construction sign 900 acquired from a LiDAR device. Also, whenthe controller recognizes the additional information, the controller maygenerate sharing data including the first subset of point data 2116 andthe additional information.

Also, the additional information may be acquired from the outside. Forexample, the controller may acquire additional information related tothe stop time of the construction sign 900 from an external server andmay generate sharing data including the additional information.

Also, the controller may transmit the sharing data to a second device.In this case, the second device may include vehicles 131 and 132, aserver 400, an infrastructure device 700, etc., but the presentinvention is not limited thereto.

FIG. 54 is a diagram illustrating a subset of point data and additionalinformation included in the content of sharing data according to anembodiment.

Referring to FIG. 54 , the first vehicle 130 may transmit sharing data3000 to the server 400. In this case, the content of the sharing datamay include the first subset of point data 2116 and the third subset ofpoint data 2118 which are related to stationary objects. Also, thecontent of the sharing data may include additional information 2300representing a stop time of a construction sign 900 represented by thefirst subset of point data 2110.

Also, although a controller of the first vehicle 130 does not acquireadditional information from the first subset of point data 2116, thecontroller may acquire additional information related to a stop timepoint of a construction site near the construction sign 900 when thecontroller acquires sensor data related to the construction site. Indetail, when class information of a plurality of subsets of point datarepresenting a worker and an excavator included in the construction siteis determined as “construction site,” the controller may acquireadditional information including a construction completion time point ofthe construction site. In this case, the construction completion timepoint may refer to stop time points of a plurality of objects related tothe construction site. Thus, the controller may generate sharing dataincluding the additional information and transmit the generated sharingdata to a second device.

4.1.3.3. Selective Sharing of Sensor Data According to Other Embodiment

Information regarding an immovable object may be prestored in a devicefor generating a high-definition map. In this case, the device fortransmitting the sensor data may select only data related to movableobjects from the sensor data and transmit the data to the device forgenerating the high-definition map.

FIG. 55 is a flowchart illustrating a method of sharing sensor datarelated to a movable object according to an embodiment.

Referring to FIG. 55 , a controller included in a first device mayobtain a set of point data through at least one sensor (S5028). In thiscase, the first device may include a vehicle, a server, aninfrastructure device, a mobile device, etc., but the present inventionis not limited thereto.

Also, the controller may determine class information of a plurality ofsubsets of point data included in the set of point data (S5029).

Also, the controller may determine the movability of a plurality ofobjects represented by the plurality of subsets of point data on thebasis of the class information (S5030).

Also, when the controller determines that a first object may movebecause class information of the first object is related to a movableobject, the controller may generate sharing data including a subset ofpoint data representing at least a portion of the first object (S5031).In this case, the content of the sharing data may include property dataof the subset of point data and may further include property data of asubset of point data representing at least a portion of a second objectrelated to an immovable object.

Also, the controller may transmit the sharing data to a second device(S5032). In this case, the second device may include a vehicle, aserver, an infrastructure device, a mobile device, etc., but the presentinvention is not limited thereto.

4.1.3.4. Selective Sharing of Sensor Data According to Still OtherEmbodiment

Also, a controller of a second device, which receives sharing data froma first device, may determine whether to store the sharing dataaccording to class information of a subset of point data included in thesharing data.

FIG. 56 is a diagram illustrating a method of selectively storingsharing data according to an embodiment.

Referring to FIG. 56 , a first device may acquire a set of point datathrough at least one sensor (S5033).

Also, a controller included in the first device may transmit sharingdata including the set of point data to a second device through at leastone communication module. In this case, the content of the sharing datamay further include additional information for the second device tofacilitate coordinate system alignment. For example, the additionalinformation may include sampling rate-related information, resolutioninformation, etc. of a sensor of the first device, but the presentinvention is not limited thereto.

Also, when the sharing data is received, a controller of the seconddevice may determine class information of a plurality of subsets ofpoint data included in the set of point data (S5035).

Also, the controller of the second device may determine whether to storedata included in the sharing data on the basis of the class information(S5036).

As an example, when the class of an object included in the classinformation is a class in which personal information needs to beprotected, the controller of the second device may generate and storeprivacy protection data obtained by processing at least a portion of theset of point data. In this case, the controller of the second device maydelete rather than store a subset of point data representing the objecthaving the class in which personal information needs to be protected.

As another example, the controller of the second device may determinethe movability of an object on the basis of class information and maystore sensor data representing an object that cannot move. In detail,the controller of the second device may store a subset of point datahaving class information related to an immovable object or property dataof this subset of point data among the plurality of subsets of pointdata included in the set of point data.

As another example, the controller of the second device may determinethe movability of an object on the basis of class information and maystore sensor data representing an object that may move. In detail, thecontroller of the second device may store a subset of point data havingclass information related to a movable object or property data of thissubset of point data among the plurality of subsets of point dataincluded in the set of point data.

Also, the second device may determine whether to store data included inthe sharing data according to whether information regarding an objectrepresented by a subset of point data included in the content of thereceived sharing data is stored in the second device.

Also, the second device may receive the sharing data and generate ahigh-definition map. In this case, when information related to immovableobjects is stored in the high-definition map of the second device, thesecond device may receive sensor data related to the movable object andupdate the high-definition map. However, the present invention is notlimited thereto. In order to update the high-definition map with theinformation related to immovable objects, the second device may receivethe sensor data related to immovable objects. In this case, the sensordata may include a set of point data, a plurality of subsets of pointdata, and property data of the plurality of subsets of point data, butthe present invention is not limited thereto.

Also, the second device may receive sharing data including privacyprotection data and match the privacy protection data to thehigh-definition map.

4.2. Selective Sharing of Sensor Data According to Occurrence of Event4.2.1. Necessity of Selective Sharing According to Occurrence of Event

A data sharing system according to an embodiment may include a firstdevice and a second device as data sharing entities. Here, the firstdevice may transmit sharing data to the second device or a server, butthe present invention is not limited thereto.

In this case, when the first device shares all acquired sensor data withthe second device or the server, various problems such as poor datasharing efficiency may occur. For example, when a set of point dataincluded in the sensor data is shared without any processing, a datastorage capacity problem, a communication server overload problem, orthe like may occur, but the present invention is not limited thereto.

In order to solve the above problems, a controller of the first devicemay generate the content of the sharing data at least partiallydifferently depending on whether an event has occurred.

For example, the controller may generate and transmit first sharing dataincluding property data before the event occurs. In this case, the eventmay include a traffic event related to vehicle driving, an environmentalevent such as rain and snow, and a regulatory event such as entry into achild protection zone, but the present invention is not limited thereto.The event will be described in detail below.

Also, the controller may generate and transmit second sharing dataincluding a set of point data or a plurality of subsets of point data inorder to transmit accurate information related to the event after theevent occurs. In this case, the second sharing data may include a set ofpoint data or a plurality of subsets of point data which have beenacquired for a predetermined time before and after the event occurs.

4.2.2. Selective Sharing Method (1) of Sensor Data According toEmbodiment

FIG. 57 is a flowchart illustrating a selective sharing method forsensor data according to another embodiment.

Referring to FIG. 57 , a controller of a first device may acquire a setof point data through at least one sensor (S5037). In this case, thefirst device may include a vehicle, an infrastructure, etc., but thepresent invention is not limited thereto.

Also, the controller may determine property data of a plurality ofsubsets of point data included in the set of point data (S5038). In thiscase, the property data may include class information, center positioninformation, size information, movement information, shape information,identification information, and the like of the subsets of point data,but the present invention is not limited thereto.

Also, the controller may generate first sharing data including theproperty data and transmit the first sharing data to a second device(S5039, S5040). In this case, the second device may include a vehicle, aserver, an infrastructure device, a mobile device, etc., but the presentinvention is not limited thereto.

Also, the controller may determine the occurrence of an event (S5041).In this case, the event may include a traffic event related to drivingand accident of vehicle, but the present invention is not limitedthereto.

Also, the controller may generate and transmit second sharing dataincluding a plurality of sets of point data acquired for a first timeperiod before and after the event occurs (S5042).

Hereinafter, a method of determining the occurrence of an eventaccording to an embodiment will be described in detail.

4.2.2.1. Method of Generating Sharing Data and Determining Occurrence ofEvent

FIG. 58 is a diagram showing a situation in which a first vehicleacquires sensor data before an event occurs according to an embodiment.

FIG. 59 is a diagram schematically showing a set of point data includedin the sensor data acquired according to FIG. 58 in a 2D plane.

Referring to FIGS. 58 and 59 , a first vehicle 133 may include a set ofpoint data 2104 including a first subset of point data 2119 representingat least a portion of a second vehicle 134 and a second subset of pointdata 2120 representing at least a portion of a third vehicle 135.

Also, the controller may determine a plurality of pieces of propertydata of a plurality of subsets of point data included in the set ofpoint data. In this case, the plurality of pieces of property data mayinclude at least one of center position information, size information,class information, shape information, movement information, oridentification information of the plurality of subsets of point data2119 and 2120, but the present invention is not limited thereto.

Also, the first device may generate first sharing data and transmit thegenerated first sharing data to the second device (S5039, S5040).

FIG. 60 is a diagram illustrating first sharing data transmitted by afirst vehicle before an event occurs according to an embodiment.

Referring to FIG. 60 , a controller of the first vehicle may generatefirst sharing data 3000 a including first property data 2206 of thefirst subset of point data and second property data 2207 of the secondsubset of point data and transmit the first sharing data 3000 a to thesecond vehicle 134.

Also, the controller may determine the occurrence of an event. Forexample, the controller may determine a traffic event 6100 between thesecond vehicle 134 and the third vehicle 135. In this case, the trafficevent 6100 may be related to at least one of an accident situationrelated to the first vehicle 133 or accident situation related to othervehicles 134 and 135 near the first vehicle.

FIG. 61 is a diagram showing a situation in which a first vehicleacquires sensor data when an event occurs according to an embodiment.

FIG. 62 is a diagram schematically showing a set of point data includedin the sensor data acquired according to FIG. 61 in a 2D plane.

Referring to FIGS. 61 and 62 , a controller of the first vehicle 133 mayacquire a second set of point data 2105 including the vehicles 134 and135 related to the traffic event 6100 through at least one sensor. Inthis case, the set of point data 2105 may include a third subset ofpoint data 2121 representing at least a portion of the second vehicle134 and a fourth subset of point data 2122 representing at least aportion of the third vehicle 135.

As an example, the controller may determine the occurrence of the eventon the basis of at least a portion of a set of point data or propertydata of the subset of point data (S5041). In detail, the controller maydetermine the occurrence of the event on the basis of at least a portionof a plurality of pieces of information included in a plurality ofpieces of property data or location information of objects included in aplurality of subsets of point data.

As a specific example, the controller of the first vehicle 133 maydetermine that the traffic event 6100 has occurred between the secondvehicle 134 and the third vehicle 135 when point data included in thethird subset of point data 2121 representing at least a portion of thesecond vehicle 134 at least partially overlaps point data included inthe fourth subset of point data 2122 representing at least a portion ofthe third vehicle 13 and also when a distance between the third subsetof point data 2121 and the fourth subset of point data 2122 isdetermined to be less than or equal to a predetermined distance on thebasis of distance information determined through the controller. In thiscase, at least one of a plurality of subsets of point data 2121 and 2122included in a set of point data the first vehicle acquires may representat least a portion of the vehicles 134 and 135 related to the event.

Also, when the plurality of subsets of point data or a plurality ofpieces of property data partially overlap each other in a 3D point datamap generated based on the set of point data, the controller maydetermine that the traffic event 6100 has occurred between the secondvehicle 134 and the third vehicle 135.

However, the present invention is not limited thereto, and thecontroller may determine the occurrence of an event even when a subsetof point data representing an event related to the event is not includedin the set of point data.

As an example, when information for determining the occurrence of theevent is included in the set of point data, the controller may determinethe occurrence of the event on the basis of the information fordetermining the occurrence of the event. As a specific example, when asubset of point data representing an object for indicating an accidentsite is included in the set of point data, the controller may determinethe occurrence of an event on the basis of the subset of point datarepresenting the object for indicating the accident site.

However, the present invention is not limited thereto, and thecontroller may determine the occurrence of an event by acquiringinformation including the occurrence of the event from the second deviceor the third device. In this case, the third device may include vehicles134 and 135, a server 400, an infrastructure device 700, etc., but thepresent invention is not limited thereto.

For example, when the server 400 determines the occurrence of the event,the server 400 may transmit the information including the occurrence ofthe event to a device near where the event has occurred. As a specificexample, when the server 400 determines that the traffic event 6100 hasoccurred, the server 400 may transmit information including theoccurrence of the traffic event 6100 to the first vehicle 133 which islocated near where the traffic event 6100 has occurred. In this case,when the information including the occurrence of the traffic event isreceived, the first vehicle 133 may determine that the traffic event6100 has occurred.

However, the present invention is not limited thereto, and thecontroller may determine the occurrence of an event by acquiring datarequest information from at least one of the second device or the thirddevice. In this case, the request information may include informationindicating the occurrence of the event.

For example, when the server 400 transmits request information forrequesting data related to the traffic event 6100 to the first vehicle133, the request information includes the information indicating theoccurrence of the traffic event 6100, and thus the first vehicle 133 maydetermine that the traffic event 6100 has occurred when the requestinformation is received.

Also, the controller may generate second sharing data 3000 b includingthe second set of point data 3100 (S5042).

FIG. 63 is a diagram illustrating second sharing data transmitted by afirst vehicle after an event occurs according to an embodiment.

Referring to FIG. 63 , a controller of the first vehicle may generateand transmit second sharing data 3000 b including the second set ofpoint data to the second vehicle 134. In this case, the second set ofpoint data may include a third subset of point data 2121 representing atleast a portion of the second vehicle 134 and a fourth subset of pointdata 2122 representing at least a portion of the third vehicle 134.

In this case, the content of the second sharing data may be at leastpartially different from the content of the first sharing data. As anexample, when the sharing data 3000 a and 300 b are received, the seconddevice needs more accurate data related to the traffic event 6100, andthus the second sharing data 3000 b may include a plurality of subsetsof point data 2121 and 2122 acquired after the traffic event 6100occurs. As another example, the resolution of a sensor for acquiringsensor data included in the content of the second sharing data may bedifferent from the resolution of a sensor for acquiring sensor dataincluded in the content of the first sharing data. For example, theresolution of the sensor for acquiring sensor data included in thecontent of the second sharing data may be higher than the resolution ofthe sensor for acquiring sensor data included in the content of thefirst sharing data, but the present invention is not limited thereto.

Also, the content of the second sharing data may include a plurality ofsets of point data acquired for a first time period before and after thetraffic event 6100 occurs. In detail, the plurality of sets of pointdata may include a set of point data acquired before the traffic event6100 occurs as well as a set of point data acquired from a sensor of thefirst vehicle 133 after the traffic event 6100 occurs. This may be toobtain accurate information related to the cause of the traffic event6100 through the data acquired before and after the traffic event 6100occurs.

However, the present invention is not limited thereto, and the contentof the second sharing data may further include property data related tothe event.

It will be appreciated that the selective sharing method for sensor dataaccording to an embodiment is not limited to the operations shown inFIG. 57 . For example, the first device may not generate sharing databefore the first device determines that an event has occurred. Thus, thefirst device may not share data with a second device before the firstdevice determines that an event has occurred.

4.2.2.2. Data Sharing Entity

Also, the first device may transmit the second sharing data (S5042).

In this case, an entity receiving the second sharing data may include avehicle, an infrastructure device, a mobile device, etc., but thepresent invention is not limited thereto. Also, the first device maytransmit the second sharing data to a second device which hastransmitted the first sharing data.

Referring to FIG. 61 again, the controller of the first vehicle 133 maytransmit the second sharing data to the second vehicle 134 related tothe traffic event 6100.

However, the present invention is not limited thereto, and when requestinformation for requesting data related to the traffic event 600 isacquired from the server 400, the controller may transmit the secondsharing data 3000 b to the server 400.

It will be appreciated that when information related to a sharing datareceiving entity is included in the request information, the controllermay transmit the second sharing data on the basis of the informationrelated to the sharing data receiving entity. For example, wheninformation regarding a sharing data receiving entity and instructing totransmit sharing data to the third vehicle 135 is included in therequest information received from the server 400, the controller maytransmit the second sharing data 3000 b to the third vehicle 135.

4.2.2.3. Generation Time of Sharing Data

When the controller of the first device determines the occurrence of anevent, the controller may generate sharing data at certain intervalsafter the event occurs. In this case, the content of the sharing datamay include at least one set of point data acquired before the eventoccurs. However, the present invention is not limited thereto, and thecontent of the sharing data may include at least one set of point dataacquired after the event occurs. In this case, the controller maytransmit the sharing data to the second device each time the sharingdata is generated.

Also, the controller may generate the sharing data after the completionof a first time period including a time point at which the event occurs.In this case, the content of the sharing data may include a plurality ofsets of point data acquired for a first time period before and after theevent occurs. In this case, the controller may transmit the sharing datato the second device after the sharing data is generated.

For example, referring to FIG. 63 again, the first vehicle 133 maytransmit second sharing data 3000 b to the second vehicle 134. In thiscase, the second sharing data 3000 b may be generated at regularintervals after the traffic event 6100 occurs. In this case, the contentof the second sharing data may include a set of point data or aplurality of subsets of point data 2121 and 2122 which are acquired whenthe traffic event occurs. Also, the content of the second sharing datamay include a plurality of sets of point data acquired before thetraffic event occurs and may include a plurality of sets of point dataacquired after the traffic event occurs.

However, the present invention is not limited thereto, and the secondsharing data 3000 b may be generated after the completion of the firsttime period before and after the traffic event 6100 occurs. In thiscase, the content of the second sharing data may include a plurality ofsets of point data acquired for a first time period including apredetermined time before and after the traffic event. However, thepresent invention is not limited thereto, and the content of the secondsharing data may include a set of point data and a plurality of subsetsof point data 2121 and 2122 which are acquired when the traffic eventoccurs.

However, the present invention is not limited thereto, and the sharingdata may be generated at the same time as the sensor data is acquired.It will be appreciated that the sharing data may be generated at anytime regardless of when the sensor data is acquired.

4.2.2.4. Various Examples of Event

The event 6000 may refer to all situational conditions related to theinside and outside of the first device. For example, the event mayinclude a traffic event, an environmental event, a regulatory event, ablind spot discovery, a user input reception, etc., but the presentinvention is not limited thereto.

For example, the event may be a traffic event related to at least one ofan accident situation related to the first device or an accidentsituation related to another device near the first device, anenvironmental event related to the surrounding environment of the firstdevice, a regulatory event related to regulations on the first device oranother device near the first device, etc., but the present invention isnot limited thereto.

Also, it will be appreciated that the above-described embodiments of theselective sharing method for sensor data are applicable to various typesof events.

In this case, the traffic event may be related to at least one of anaccident situation related to the first vehicle or accident situationsrelated to other vehicles near the first vehicle. For example, thetraffic event may include a vehicle accident, an accident between avehicle and a pedestrian, a traffic jam, etc., but the present inventionis not limited thereto.

FIG. 64 is a diagram illustrating a situation in which a traffic eventhas occurred according to an embodiment.

Referring to FIG. 64 , a chain collision accident may be included in thetraffic event 6100. In this case, the content of sharing data that isshared between data sharing entities may vary before and after thetraffic event 6100 occurs. For example, before the traffic event 6100occurs, sharing data including property data of a subset of point datamay be shared, but after the traffic event 6100 occurs, sharing dataincluding at least one of a set of point data or a subset of point datamay be shared.

Also, the environmental event may be related to the surroundingenvironment of the first device. For example, the environmental eventmay include occurrence of bad weather, deterioration of road conditions,sudden rain or snow, occurrence of fog or sea fog, etc., but the presentinvention is not limited thereto.

FIG. 65 is a diagram illustrating a situation in which an environmentalevent has occurred according to an embodiment.

Referring to FIG. 65 , rain that suddenly falls in an area where avehicle is traveling may be included in an environmental event 6200. Inthis case, the content of sharing data that is shared between datasharing entities may vary before and after the environmental event 6200occurs. For example, before the environmental event 6200 occurs, sharingdata including property data of a subset of point data may be shared,but after the environmental event 6200 occurs, sharing data including atleast one of a set of point data or a subset of point data may beshared.

For example, when it suddenly rains while a vehicle is traveling, it maybe difficult for at least one sensor placed in the vehicle to acquireaccurate sensor data for a plurality of objects located near thevehicle. Accordingly, in order to share more accurate sensor data, thevehicle and other devices may generate sharing data including at least aportion of the set of point data or the subset of point data and sharethe generated sharing data.

As another example, the regulatory event may be related to regulationson the first device or other devices near the first device. For example,the regulatory event may include entry into a child protection zone,entry into a speed enforcement zone, approval for data sharing by anexternal server, entry into an available communication zone, etc., butthe present invention is not limited thereto.

FIG. 66 is a diagram illustrating a situation in which a regulatoryevent has occurred according to an embodiment.

Referring to FIG. 66 , a situation in which a traveling vehicle enters achild protection zone may be included in a regulatory event 6300. Inthis case, the content of sharing data shared between data sharingentities may vary before and after the regulatory event 6300 occurs. Forexample, before the regulatory event 6300 occurs, sharing data includingproperty data of a subset of point data may be shared, but after theregulatory event 6300 occurs, sharing data including at least one of aset of point data or a subset of point data may be shared.

For example, when a vehicle enters a child protection zone, it may bedifficult for the vehicle to avoid a collision with a pedestrian whosuddenly runs onto a road. Accordingly, in order to share accurateinformation on at least one object included in sensor data acquired fromthe vehicle and other vehicles, the vehicle and the other devices maygenerate sharing data including a subset of point data or a set of pointdata representing the at least one object and then share the generatedsharing data.

Also, in order to acquire information on an object not located in thefield of view of at least one sensor placed in the vehicle or an objectnot included in sensor data acquired from the at least one sensor, thevehicle may receive sensor data from at least one infrastructure devicelocated in a child protection zone after the vehicle enters the childprotection zone.

As another example, the event may include a sensor failure event. Indetail, when at least one sensor included in an autonomous drivingvehicle fails while the vehicle is traveling, the content of sharingdata which is shared between the autonomous driving vehicle and otherdevices may vary before and after the sensor fails.

4.2.3. Selective Sharing Method (2) of Sensor Data According toEmbodiment

FIG. 67 is a diagram illustrating a method of requesting, by a server,data regarding an event or indicating that an event has occurredaccording to an embodiment.

Referring to FIG. 67 , the server may recognize an event that hasoccurred in a first region at a first time (S5043).

Details on how the server checks an event (it is noted that a trafficevent is changed to an event) have been described in Section 4.2.2.1.,and thus will be omitted here.

Also, the first time may refer to a representative time related to theoccurrence of the event. For example, the first time may refer to a timeat which the event actually occurs. However, the present invention isnot limited thereto, and the first time may refer to a time at which theserver recognizes the event.

Also, the first region may refer to a representative region related tothe occurrence of the event. For example, the first region may refer toa region including all objects related to the event. However, thepresent invention is not limited thereto, and when the event is afender-bender, the first region may refer to a point where a minorcollision between occurs or a predetermined region including the pointwhere the minor collision occurs.

Also, the server may transmit a first message for requesting sensor datarelated to the event to a first device (S5044).

Also, the server may transmit a second message indicating that the eventhas occurred to a second device (S5045).

Also, the server may receive sensor data related to the event from thefirst device (S5046).

The operation of transmitting a message and receiving sharing data amongthe above-described operations included in the server operation methodwill be described below.

4.2.3.1. Message Transmission Range

When an event is recognized, a server may request data from a firstdevice located near a first area where the event has occurred. In thiscase, the server may request sensor data from the first device or mayrequest various types of data other than the sensor data.

FIG. 68 is a diagram showing a situation in which a server and a vehiclecommunicate with each other to share data according to an embodiment.

Referring to FIG. 68 , a first vehicle 136 may be located in a firstrange from a region where the event 6100 has occurred and may acquiresensor data related to a traffic event 6100 through at least one sensor.

Also, when the first vehicle 136 is located in a first range 7100 from afirst region where the traffic event 6100 has occurred, a server 400which has recognized the traffic event 6100 may transmit a first messagerequesting sensor data to the first vehicle 136.

In this case, the first range 7100 may correspond to a region includedin the inside of a predetermined shape based on the first region. Forexample, the first range may be a region included in the inside of anirregular shape, a circle, a polygonal shape, or the like, but thepresent invention is not limited thereto.

Also, the first range 7100 may be determined based on sensor data. Indetail, when an object related to the traffic event is included insensor data acquired by the first device located in the first range, thefirst range may be set such that the first device is located in thefirst range from the first region.

Also, the first range 7100 may include a first sub-range and a secondsub-range.

FIG. 69 is a diagram illustrating a first sub-range included in a firstrange according to an embodiment.

Referring to FIG. 69 , the first range 7100 may include the inside of asphere with respect to the region where the traffic event 6100 hasoccurred.

Also, a fourth vehicle 139 may be located in the first sub-range 7110included in the first range 7100. In this case, the first sub-range 7110may correspond to a region in which information related to the trafficevent 6100 can be acquired in the first range 7100. In detail, when thefourth vehicle 139 is located in the first sub-range 7110, the fourthvehicle 139 may acquire data regarding the traffic event 6100 through atleast one sensor.

Also, the first sub-range 7110 may be determined based on sensor data.In detail, when an object related to the traffic event is included insensor data acquired by the fourth vehicle 139 located in the firstrange 7100, the first sub-range 7110 may be set such that the fourthvehicle 139 is located in the first sub-range 7110 from the firstregion. In this case, the sensor data acquired by the fourth vehicle 139may include a subset of point data representing at least a portion ofthe object related to the traffic event 6100.

Also, the third vehicle 138 may be located in a second sub-range 7120included in the first range 7100. In this case, the second sub-range7120 may correspond to a region in which information related to thetraffic event 6100 cannot be acquired in the first range 7100. Indetail, when the third vehicle 138 is located in the second sub-range7120, the third vehicle 138 may acquire data regarding the traffic event6100 through at least one sensor.

Also, the second sub-range 7120 may be determined based on sensor data.In detail, when an object related to the traffic event is not includedin sensor data acquired by the third vehicle 138 located in the firstrange 7100 or when the sensor data and the object related to the trafficevent have a low correlation, the second sub-range 7120 may be set suchthat the third vehicle 183 is located in the second sub-range from thefirst region. In this case, the sensor data acquired by the thirdvehicle 138 may not include a subset of point data representing at leasta portion of the object related to the traffic event 6100.

Also, the server may notify the second device located near the regionwhere the traffic event has occurred of the occurrence of the event.

Referring to FIG. 68 again, the second vehicle 137 may be located in asecond range 7200 from the first region where the traffic event 6100 hasoccurred.

Also, when the second vehicle 137 is located in the second range 7200,which represents a predetermined region outside the first range 7100,from the first region where the traffic event 6100 has occurred, theserver may transmit a second message indicating that the traffic eventhas occurred to the second vehicle 137.

In this case, the second range 7200 may correspond to a region includedin the inside of a predetermined shape with respect to the first region.For example, the second range may be a region included in the inside ofan irregular shape, a circle, a polygonal shape, or the like in theregion outside the first range 7100, but the present invention is notlimited thereto.

Referring to FIG. 68 again, a path of the second vehicle 137 may berelated to the first region where the traffic event 6100 has occurred.In detail, when the path of the second vehicle 137 located in the secondrange 7200 from the first region is related to the first region relatedto the traffic event 6100, the server 400 may transmit a second messageindicating that the traffic event has occurred to the second vehicle137.

Also, the second range 7200 may be determined based on the path of thesecond vehicle 137. In detail, when the path of the second vehicle 137is related to the first region where the traffic event 6100 hasoccurred, the server 400 may determine the second range 7200 such thatthe second vehicle 137 is located in the second range 7200.

Also, the second range may include the first range. In this case, theserver may transmit the first message and the second message to avehicle located in the first range.

4.2.3.2 Reception of Sharing Data

Also, the server may receive sensor data from the first device inresponse to the first message. In this case, the sensor data may includea set of point data, a subset of point data, property data of the subsetof point data, etc., but the present invention is not limited thereto.

FIG. 70 is a diagram illustrating data included in the sharing datatransmitted by a first vehicle to a server according to an embodiment.

Referring to FIG. 70 , the first vehicle 136 included in the first rangemay transmit sharing data 3000 to the server 400 in response to thefirst message. In this case, the content of the sharing data may includea first set of point data 2106 acquired at a first time point at whichthe traffic event 6100 occurs.

Also, the content of the sharing data may include a plurality of sets ofpoint data acquired for a first time period including a first time atwhich the traffic event occurs in order to share information regardingbefore and after the occurrence of the traffic event 6100. In this case,the plurality of sets of point data may include the first set of pointdata 2106.

Also, details on when the sharing data is generated have been describedin Section 4.2.2.3., and thus will be omitted here.

Also, a server 400 which has received the sharing data may reconfigurethe traffic event on the basis of a plurality of sets of point dataincluded in the content of the sharing data. In detail, the server 400may reconfigure the traffic event by listing, in chronological order, aplurality of sets of point data related to the traffic event acquiredfor the first time period. Also, the server 400 may reconfigure thetraffic event by re-sampling a plurality of sets of point data relatedto the traffic event acquired for the first time period.

The scheme of reconfiguring the traffic event can be used by thoseskilled in the art, and thus a detailed description thereof will beomitted here.

Also, the reconfigured traffic event may be transmitted to at least onevehicle and displayed to an occupant through an infotainment system ofthe at least one vehicle. However, the present invention is not limitedthereto, and the reconfigured traffic event may be transmitted to anexternal institution.

4.2.3.3. Information Included in Message

FIG. 71 is a diagram illustrating information included in a firstmessage according to an embodiment.

Referring to FIG. 71 , a first message 1431 received from a serverthrough a message window 1430 included in at least one infotainmentsystem of a vehicle may be displayed.

Also, the first message 1431 may include time information related to theoccurrence time of the event. In this case, the time information mayinclude first information representing that the event has occurred at afirst time. Also, a controller of the vehicle may recognize that theevent has occurred at at least one of a time point at which the firstmessage 1431 is received, a time point at which the time information isacquired, or a time point at which the first information is acquired.

Also, the first message 1431 may include request information for datarelated to the event.

Also, the controller of the vehicle may receive an input from anoccupant in the vehicle in response to the first message 1431. In thiscase, the controller may receive an input for accepting the transmissionof data related to the event from the occupant or may receive an inputfor rejecting the transmission of data related to the event.

When the controller receives the input for accepting the transmission ofthe data related to the event, the controller may generate sharing dataincluding at least one subset of point data representing at least aportion of an object related to the event and may transmit the sharingdata to the server or the object related to the event.

FIG. 72 is a diagram illustrating information included in a secondmessage according to an embodiment.

Referring to FIG. 72 , a second message 1432 received from a serverthrough a message window 1430 included in at least one infotainmentsystem of a vehicle may be displayed.

Also, the second message 1432 may include position information relatedto the occurrence position of the event. In this case, the positioninformation may include second information representing that the eventhas occurred in a first region. Also, a controller of the vehicle mayrecognize that the event has occurred at at least one of a time point atwhich the second message 1432 is received, a time point at which theposition information is acquired, or a time point at which the secondinformation is acquired.

Also, the second message 1432 may include at least a portion ofinformation included in the first message. For example, the secondmessage 1432 may include time information representing that the eventhas occurred at a first time, but the present invention is not limitedthereto.

Also, in some embodiments, a server which has recognized the occurrenceof an event may transmit a message requesting that data should becontinuously shared between a device related to the event and a devicelocated near the vehicle. For example, when a server recognizes that anenvironmental event, such as sudden rain, has occurred, the server maytransmit a message requesting that data should be continuously sharedbetween a plurality of vehicles in relation to the environmental event.

Also, in some embodiments, the server may recognize that a sensorfailure event has occurred in an autonomous vehicle where at least onesensor is placed. In this case, in order to prevent the risk of anaccident of the autonomous vehicle that may occur due to a sensorfailure, the server may transmit a message requesting that data shouldbe shared with the autonomous vehicle to a vehicle located near theautonomous device.

4.2.4. Selective Sharing Method (3) of Sensor Data According toEmbodiment

FIG. 73 is a diagram illustrating an example related to a selectivesharing method for sensor data depending on the range.

Referring to FIG. 73 , a second device and a third device may acquire aset of point data using at least one sensor (S5047).

Also, the second device, which is located in a third range included inan available communication range from a region where the traffic eventhas occurred, may transmit first sharing data including a set of pointdata to a first device (S5048).

In this case, the third range may refer to a range in which data relatedto the traffic event can be acquired. Also, the third range may bedetermined in the same manner as the above-described first rangedetermination scheme included in Section 4.2.3.1.

Also, the content of the first sharing data may include the set of pointdata, but the present invention is not limited thereto. The content ofthe first sharing data may include at least one of the set of pointdata, at least one subset of point data included in the set of pointdata, or property data of the at least one subset of point data, andinformation regarding the second device, but the present invention isnot limited thereto.

Also, a third device, which is located in an available communicationrange from the region where the traffic event has occurred and islocated outside the third range, may transmit second sharing dataincluding position information of the third device to the first device(S5049).

In this case, the available communication range may refer to apredetermined region where it is possible to communicate with an objectrelated to the traffic event to share data. For example, the availablecommunication range may include a region where a vehicle related to thetraffic event can communication with other devices through a V2X system.

Also, the content of the second sharing data may include positioninformation of the third device, but the present invention is notlimited thereto. The content of the second sharing data may includebasic information regarding the third device. In this case, the positioninformation of the third device may include GPS information of the thirddevice. Also, the position information of the third device may includethe position coordinates of the third device which are acquired from atleast one sensor included in a fourth device located near the thirddevice.

4.2.5. Selective Sharing Method (4) of Sensor Data According toEmbodiment

FIG. 74 is a diagram illustrating a selective data sharing methodaccording to a blind spot during the driving of a vehicle in relation toa regulatory event according to an embodiment.

Referring to FIG. 74 , a first device (e.g., a vehicle) may enter aspecific regulation region such as a child protection zone (S5050).Here, the specific regulation region may refer to a region to whichlegal or customary regulations are applied in the first device comparedto other regions. For example, the child protection zone may refer to aregion where the driving speed of a vehicle is regulated to apredetermined speed or less and in which special attention is requiredfor the safety of pedestrians including children in order to protectchildren from the vehicle. Thus, the first device may need to moreaccurately recognize the positions or movements of nearby pedestrians inthe child protection zone than in other zones.

Also, the first device may request a second device located in the childprotection zone to determine whether a blind spot where an object cannotbe recognized is in the field of view of a sensor of the first device.

Also, the second device (e.g., an infrastructure device) located in thechild protection zone may notify the first device that the first devicehas entered the child protection zone (S5051). In this case, the methodof the second device notifying the first device that the first devicehas entered the child protection zone may include transmitting anotification message indicating that the above-described regulatoryevent has occurred, but the present invention is not limited thereto.However, the present invention is not limited thereto, and the firstdevice may notify the second device that the first device has enteredthe child protection zone.

Also, when the first device enters the child protection zone, the seconddevice may transmit first sharing data to the first device (S5052). Inthis case, the content of the first sharing data may include sensor dataacquired from at least one sensor placed in the second device, dataother than the sensor data, etc., but the present invention is notlimited thereto. For example, the sensor data may include a set of pointdata, a subset of point data, property data of the subset of point data,etc., but the present invention is not limited thereto.

Also, the second device may detect a blind spot related to the firstdevice (S5053). In this case, the method of the second device detectinga blind spot related to the first device may include various methods.

As an example, the first device may detect a blind spot related to thefirst device by itself and transmit blind spot-related information tothe second device. As a specific example, when the first device is avehicle, the vehicle may compare a high-definition map received from theoutside to sensor data acquired from at least one sensor placed in thevehicle and may determine that a blind spot is present when an objectthat is not included in the sensor data is included in thehigh-definition map. In this case, the vehicle may transmit informationrelated to the presence of a blind spot to the second device.

However, the present invention is not limited thereto, and the firstdevice may detect a blind spot on the basis of a ratio of ground-relateddata to non-ground data in sensor data acquired through at least onesensor placed in the first device. In detail, when the proportion of thenon-ground data covered by an object included in the sensor dataacquired by the first device is greater than or equal to a predeterminedproportion, the first device may determine that a blind spot is presentand may transmit information related to the presence of the blind spotto the second device.

As another example, when the first device enters a specific regulationregion such as a child protection zone, the second device may determinethat a blind spot related to the first device is present regardless ofwhether the blind spot related to the first device is actually present.Specifically, since the risk of collision between the first device and apedestrian is high in a specific regulation region such as a childprotection zone, the second device may determine that a blind spotrelated to the first device is present when the first device enters thechild protection zone.

However, the present invention is not limited thereto, and the seconddevice may determine the presence of a blind spot related to the firstdevice on the basis of sensor data acquired through at least one sensorplaced in the second device. In other words, the second device maydetermine the presence of the blind spot related to the first device onthe basis of a positional relationship between a plurality of objectsincluding the first device included in the sensor data acquired by thesecond device. In detail, when the second device determines that thefirst device cannot recognize a specific object included in the sensordata on the basis of position information of the first device, thesecond device may determine the presence of the blind spot related tothe first device.

Also, when the blind spot related to the first device is detected, thesecond device may transmit second sharing data to the first device. Inthis case, the content of the second sharing data may include a subsetof point data representing at least a portion of an object located inthe blind spot of the first device, but the present invention is notlimited thereto. The content of the second sharing data may includeproperty data of the subset of point data. Also, the content of thesecond sharing data may include all data included in the sensor dataacquired by the second device as well as the data regarding the objectlocated in the blind spot of the first device.

5. Processing and Use of Sharing Data 5.1. Overview

In a data sharing system according to an embodiment, a first device maytransmit sharing data including sensor data to a second device. In thiscase, the first device and the second device may include a vehicle, aserver, an infrastructure device, a mobile device, or the like, but thepresent invention is not limited thereto. In this case, the seconddevice, which has received the sharing data, may process the sensor dataincluded in the content of the sharing data, and the processed sensordata may be utilized to control the second device, an apparatusincluding the second device, or the like. For example, when the seconddevice is a LiDAR device and an apparatus including the second device isa vehicle, a controller of the LiDAR device or a controller of thevehicle may process sensor data included in the content of the sharingdata to control the LiDAR device or control the vehicle.

In the following description, for convenience of description, an entitythat implements the description in Section 5 is expressed as thecontroller of the vehicle, but the present invention is not limitedthereto. It will be appreciated that the controller of the second deviceor the controller of the apparatus including the second device may alsobe an entity that implements the description in Section 5.

Also, the content of the sharing data may include a set of point data, asubset of point data, property data, etc., but the present invention isnot limited thereto.

Also, the content of the sharing data may include data other than thesensor data. For example, the content of the sharing data may includetraffic event-related information, position information of the firstdevice, or a destination of the first device, etc., but the presentinvention is not limited thereto.

Also, the controller of the second device may process sharing datadifferently as described in Section 3.4.1. according to the type of thecontent of the received sharing data.

Also, in order to match the sensor data acquired from the first deviceincluded in the content of the sharing data to sensor data acquired fromthe second device, the controller of the second device may align acoordinate system using the scheme described in Section 3.4.2.

Also, the second device may receive sharing data from the first devicein order to acquire information regarding an object placed in a regionwhere sensor data cannot be acquired (e.g., a blind spot) in the fieldof view of at least one sensor included in the second device. Forexample, when a running vehicle enters a child protection zone, thevehicle may receive, from an infrastructure device placed in the childprotection zone, sharing data including sensor data acquired from asensor of the infrastructure device in order to acquire informationregarding an object not included in sensor data acquired from a sensorplaced in the vehicle.

5.2. Various Embodiments of Processing and Using Sensor Data and SharingData 5.2.1. Case in which Set of Point Data is Included in Sharing Data

In a data sharing system according to an embodiment, a first device maytransmit sharing data including a set of point data acquired from asensor to a second device. In this case, the second device may processthe received set of point data in the same scheme as described inSection 3.4.1.1.

For example, referring to FIGS. 36 to 38 again, an infrastructure device700 may transmit sharing data 3000 including a first set of point data3100 to a first vehicle 122. For convenience of description, when theelements shown in FIGS. 36 to 38 correspond to the elements described inSection 5.2.1, the infrastructure device 700 described with reference toFIGS. 36 to 38 may correspond to the first device described in Section5.2.1., and the first vehicle 122 described with reference to FIGS. 36to 38 may correspond to the second device described in Section 5.2.1. Inthis case, the controller of the first vehicle 122 may acquireinformation regarding an object included in a plurality of sets of pointdata using a second set of point data 2100 acquired from a sensor placedin the first vehicle 122 and a first set of point data 3100 included inthe sharing data acquired from the infrastructure device 700.

5.2.2. Case in which Property Data is Included in Sharing Data

Referring to FIG. 39 again, the sharing data 3000 may include propertydata of a subset of point data representing at least a portion of anobject. In this case, the property data may include center positioninformation, size information, shape information, movement information,identification information, etc., but the present invention is notlimited thereto.

5.2.2.1. Processing of Sharing Data and Aligning of Coordinate SystemAccording to Embodiment

FIG. 75 is a flowchart illustrating a scheme of processing property dataincluded in sharing data according to an embodiment.

Referring to FIG. 75 , a controller of a vehicle may acquire a first setof point data through at least one sensor placed in the vehicle (S5055).

Also, the controller of the vehicle may determine first property data ofa first subset of point data included in the first set of point data(S5056).

Also, the controller of the vehicle may generate first standard propertydata on the basis of the first property data (S5057).

Also, a first device may acquire a second set of point data through atleast one sensor placed in the first device (S5058).

Also, a controller of the first device may determine second propertydata of a second subset of point data included in the second set ofpoint data (S5059).

Also, the controller of the first device may transmit sharing dataincluding the second property data to the vehicle (S5060).

Also, the controller of the vehicle may generate second standardproperty data using the second property data received from the firstdevice (S5061).

Also, the controller of the vehicle may control the vehicle on the basisof the first standard property data and the second standard propertydata (S5062).

Hereinafter, the operations described with reference to FIG. 75 will bedescribed in detail.

5.2.2.1.1. Acquisition of Set of Point Data and Property Data

FIG. 76 is a diagram showing a situation in which a vehicle and aninfrastructure device acquire sensor data to perform data sharingaccording to an embodiment.

Referring to FIG. 76 , a vehicle 140 and an infrastructure device 700may acquire, through at least one sensor, sensor data includinginformation regarding at least one object placed in the field of view ofthe sensor.

In detail, a controller of the vehicle 140 may acquire a first set ofpoint data through at least one sensor placed in the vehicle and maydetermine first property data of the first subset of point datarepresenting at least a portion of a building 500 included in the firstset of point data (S5055, S5056). In this case, the first set of pointdata may not include information regarding a pedestrian 800 covered bythe building 500.

Also, the first property data (see 2208 in FIG. 77 ) may include centerposition information, size information, shape information, and the likeof the first subset of point data, but the present invention is notlimited thereto.

Also, considering the location of the infrastructure device 700, theinfrastructure device 700 may measure the pedestrian 800 and thebuilding 500 using at least one sensor. In this case, the infrastructuredevice 700 may determine a second set of point data through at least onesensor placed in the infrastructure device 700, and a controller of theinfrastructure device 700 may determine second property data of a secondsubset of point data representing at least a portion of the pedestrian800 included in the second set of point data. Also, the second set ofpoint data may include a third subset of point data representing atleast a portion of the building 500.

In this case, since the second subset of point data represents at leasta portion of the pedestrian 800 not included in the first set of pointdata, the infrastructure device 700 may transmit the second subset ofpoint data or the second property data of the second subset of pointdata to the vehicle 140 in order to prevent the risk of collision withthe pedestrian 800 that may occur while the vehicle is traveling.

5.2.2.1.2. Generation of Standard Property Data

Set of point data and property data included in sensor data acquiredfrom at least one sensor may be shown in a coordinate system based onany origin. In this case, the origin may correspond to the position ofthe sensor that has acquired the set of point data and the propertydata. For example, the origin may correspond to the optical origin of aLiDAR device that has acquired the sensor data, but the presentinvention is not limited thereto.

FIG. 77 is a diagram illustrating a method in which a controller of avehicle shows first property data and first standard property data in afirst local coordinate system and a global coordinate system,respectively, according to an embodiment.

Referring to FIG. 77 , first property data 2208 may be shown in a firstlocal coordinate system 9100 based on a first origin O₁. However, thepresent invention is not limited thereto, and the first set of pointdata and the first subset of point data may also be shown in the firstlocal coordinate system 9100.

As a specific example, when the first property data 2208 includes centerposition information of the first subset of point data, the centerposition coordinates of the first subset of point data included in thecenter position information may be shown in the first local coordinatesystem 9100.

In this case, the first origin O₁ may correspond to the position of thesensor that has acquired the first set of point data. For example, whenthe vehicle 140 acquires the first set of point data through a LiDARdevice, the first origin O₁ may correspond to the optical origin of theLiDAR device.

Also, the first origin O₁ may correspond to the position of the vehicle140. For example, a controller of the vehicle 140 may set the firstorigin O₁ on the basis of GPS position information of the first vehicle140.

Also, the first origin O₁ may correspond to the position of the centerof gravity of the vehicle 140, the position of the center of gravity ofthe sensor, or the like, but the present invention is not limitedthereto.

Also, referring to FIG. 77 again, the controller of the vehicle 140 maygenerate first standard property data 2501 on the basis of the firstproperty data 2208 (S5057).

Here, the standard property data represents data for matching thepositions of various pieces of property data to a single coordinatesystem, and the first standard property data 2501 generated based on thefirst property data 2208 and the second standard property data (see 3502in FIG. 78 ) generated based on second property data (see 3202 in FIG.78 ) may have the same origin.

In an example of FIG. 77 , the first standard property data 2501 may beshown in the global coordinate system 9200 based on a second origin O₂.In detail, the controller of the vehicle 140 may generate the firststandard property data 2501 by aligning the first property data 2208shown in the first local coordinate system 9100 with the globalcoordinate system 9200. In this case, the controller of the vehicle 140may align the first local coordinate system 9100 with the globalcoordinate system 9200 on the basis of the scheme described in Section3.4.2.

However, the present invention is not limited thereto, and thecontroller of the vehicle 140 may set the first local coordinate system9100 as a global coordinate system. In this case, the origin of thefirst local coordinate system 9100 may be the same as the origin of thelocal coordinate system. In other words, when the first local coordinatesystem 9100 is set as a global coordinate system, the position of thesecond origin O₂ may match the position of the first origin O₁.

More specifically, the controller of the vehicle 140 may set the firstlocal coordinate system 9100 as a global coordinate system based on thefirst origin O₁ without changing the position of the origin of the firstlocal coordinate system 9100.

As a specific example, when the first property data 2208 includes centerposition information of the first subset of point data, the controllerof the vehicle 140 may show, in the global coordinate system 9200, thecoordinate position coordinates of the first subset of point dataincluded in the center position information.

Also, the global coordinate system 9200 may include a predeterminedorigin. In this case, the predetermined origin may refer to the originof the coordinate system based on GPS position information.

Also, the second origin O₂ may correspond to the optical origin of aLiDAR device included in the vehicle 140.

Also, when the first local coordinate system 9100 is set as a globalcoordinate system, the position of the second origin O₂ may match theposition of the first origin O₁.

FIG. 78 is a diagram illustrating a method in which a controller of avehicle generates second standard property data on the basis of secondproperty data shown in a second local coordinate system according to anembodiment.

Referring to FIG. 78 , a controller of the infrastructure device mayshow second property data 3202 and third property data 3203 in a secondlocal coordinate system 9300 based on a third origin O₃.

Here, the second local coordinate system 9300 has a different originfrom the first local coordinate system, and the second local coordinatesystem 9300 and the first local coordinate system 9100 may have the samecoordinate system type (e.g., the second local coordinate system 9300and the first local coordinate system 9100 are Cartesian coordinatesystems) and may also have different coordinate system types (e.g., thesecond local coordinate system 9300 is a polar coordinate system, andthe first local coordinate system 9100 is a Cartesian coordinatesystem).

Also, the second property data 3202 may be determined based on a secondsubset of point data representing at least a portion of the pedestrian800 of FIG. 76 , and the third property data 3203 may be determinedbased on a third subset of point data representing at least a portion ofthe building 500 of FIG. 76 . However, the present invention is notlimited thereto, and the second set of point data, the second subset ofpoint data, and the third subset of point data may be shown in thesecond local coordinate system 9300.

In this case, the third origin O₃ may correspond to the position of thesensor that has acquired the second set of point data. For example, whenthe infrastructure device 700 acquires the second set of point datathrough a LiDAR device, the third origin O₃ may correspond to theoptical origin of the LiDAR device.

Also, the third origin O₃ may correspond to the position of theinfrastructure device 700. For example, the controller of theinfrastructure device 700 may set the third origin O₃ on the basis ofGPS position information of the infrastructure device 700.

Also, the third origin O₃ may correspond to the position of the centerof gravity of the infrastructure device 700, the position of the centerof gravity of the sensor, or the like, but the present invention is notlimited thereto.

Also, the controller of the infrastructure device 700 may transmitsharing data including the second property data 3202 to the vehicle 140(S5060).

In this case, the second property data 3202 may be included in thesecond set of point data and may be determined based on a second subsetof point data representing at least a portion of a pedestrian which isnot included in the first set of point data.

Also, the content of the sharing data may further include the thirdproperty data 3203. In this case, the third property data 3203 may bedetermined based on a third subset of point data representing at least aportion of a building included in the first set of point data and thesecond set of point data.

In some embodiments, it will be appreciated that the content of thesharing data may not include the third property data 3203.

However, the present invention is not limited thereto, and the contentof the sharing data may further include basic information of theinfrastructure device 700 or the like.

Also, referring to FIG. 78 again, the controller of the vehicle 140 maygenerate second standard property data 3502 on the basis of the secondproperty data 3202 included in the sharing data received from theinfrastructure device 700 (S5061).

In this case, the second standard property data 3502 may be shown in aglobal coordinate system 9200 based on the second origin O₂. In detail,the controller of the vehicle 140 may generate the second standardproperty data 3502 by aligning the second property data 3202 shown inthe second local coordinate system 9300 with the global coordinatesystem 9200 in which the first standard property data 2501 is shown. Inthis case, the controller of the vehicle 140 may align the second localcoordinate system 9300 with the global coordinate system 9200 on thebasis of the scheme described in Section 3.4.2.

For example, when the first local coordinate system 9100 is set as theglobal coordinate system 9200, the controller of the vehicle maygenerate the second standard property data 3502 by aligning the receivedsecond property data 3202 with the first local coordinate system 9100.

Also, in the method of processing and using sharing data according to anembodiment, which is shown in FIG. 76 , the controller of the vehiclemay determine whether an object represented by at least one piece ofproperty data included in the content of the sharing data is the same asan object represented by a first set of point data.

For example, an object represented by third property data 3203 includedin the sharing data received from the infrastructure device 700 may bethe same as the building 500 represented by the first property data. Inthis case, the controller of the vehicle 140 may generate third standardproperty data 3503 on the basis of the third property data 3203.

In this case, the third standard property data 3503 may be shown in theglobal coordinate system 9200 based on the second origin O₂. In detail,the controller of the vehicle 140 may generate the third standardproperty data 3503 by aligning the third property data 3203 shown in thesecond local coordinate system 9300 with the global coordinate system9200 in which the first standard property data 2501 is shown. In thiscase, the controller of the vehicle 140 may align the second localcoordinate system 9300 with the global coordinate system 9200 on thebasis of the scheme described in Section 3.4.2.

Also, the controller of the vehicle 140 acquires the third property data3203 or the third standard property data 3503 for the same building 500,and thus it is possible to implement a higher temporal resolution forthe building 500. In detail, by acquiring the third property data 3203or the third standard property data 3503 from the infrastructure device700, it is possible to reinforce information regarding the building 500that cannot be acquired in a certain time interval according to theframe rate of the LiDAR device placed in the vehicle 140.

However, the present invention is not limited thereto, and thecontroller of the vehicle 140 may not receive the third property data3203 from the infrastructure device 700. In detail, since a first set ofpoint data acquired by the vehicle 140 through a sensor includes a firstsubset of point data representing at least a portion of the building500, the controller of the vehicle 140 may not receive a third subset ofpoint data representing the same object and the third property data 3203determined based on the third subset of point data from theinfrastructure device 700.

Also, the controller of the vehicle 140 may not store the third propertydata 3203 received from the infrastructure device 700.

Also, when the third property data 3203 is received from theinfrastructure device 700, the controller of the vehicle 140 maygenerate the third standard property data 3503 without generating thefirst standard property data 2501.

Also, the controller of the vehicle 140 may determine whether a sensorplaced in the vehicle 140 is abnormal on the basis of the first standardproperty data 2501 and the third standard property data 3503. In detail,when the position information of the building 500 included in the thirdstandard property data 3503 generated through the above-describedcoordinate system alignment method is different from the positioninformation of the building 500 included in the first standard propertydata 2501, the controller of the vehicle 140 may determine that thesensor placed in the vehicle 140 is unfastened. Also, when it isdetermined that the sensor is unfastened, the controller of the vehicle140 may transmit a notification indicating that the sensor is unfastenedto an occupant. In this case, the notification may be displayed to theoccupant through an infotainment system. However, the present inventionis not limited thereto, and the notification may be transmitted to theoccupant through a scheme known to those skilled in the art, such assound.

Also, the sensor data processing method according to an embodiment isnot limited to the operations shown in FIG. 75 , and the controller ofthe infrastructure device may generate second standard property data onthe basis of second property data and may transmit sharing dataincluding the second standard property data to the vehicle.

However, the present invention is not limited thereto, and when thecontroller of the vehicle receives a high-definition map including thesecond property data, the controller of the vehicle may set the secondlocal coordinate system as a global coordinate system. In this case, inorder to match sensor data acquired from the sensor placed in thevehicle to the high-definition map, the controller of the vehicle mayalign a first local coordinate system in which the first property datais shown with the global coordinate system.

5.2.2.1.3. Vehicle Control Using Standard Property Data—Path Generation(Path Planning)

A controller of a vehicle may control the vehicle on the basis of aplurality of pieces of standard property data. However, the presentinvention is not limited thereto, and the controller of the vehicle maycontrol the vehicle on the basis of at least one of a set of point data,a subset of point data, and property data.

For example, the controller of the vehicle may control the vehicle usingsensor data or sharing data as described in Section 2.3. or Section 3.5.As a specific example, the controller of the vehicle may match theplurality of pieces of standard property data to a high-definition map,control the speed and direction of the vehicle, or control the path ofthe vehicle.

In this case, the path of the vehicle may include a global path and alocal path. Here, the global path may refer to a path to a destinationof the vehicle which is generated based on GPS position information, butthe present invention is not limited thereto. Also, the local path mayrefer to a path that is generated based on sensor data acquired from asensor placed in the vehicle or sharing data, but the present inventionis not limited thereto. As an example, one global path may correspond toa plurality of local paths and also may be generated by adding aplurality of local paths. However, the present invention is not limitedthereto, and a global path and a local path may be formed independently.

Also, the global path or the local path may include the direction of thevehicle, the speed of the vehicle, etc. In detail, the global path orthe local path may include the position of the vehicle, a direction inwhich the vehicle is to travel, the traveling speed of the vehicle,etc., but the present invention is not limited thereto.

FIG. 79 is a diagram illustrating a global path according to anembodiment.

Referring to FIG. 79 , the controller of the vehicle may generate andshow a global path 8000 in a high-definition map. In this case, thecontroller of the vehicle may control the vehicle to travel along theglobal path 8000.

In this case, the controller of the vehicle may generate a global path8000 along which the vehicle is to travel on the basis of the locationand destination of the vehicle before the vehicle starts to travel.

Also, when an input for an origin and a destination of an occupant isreceived, the controller of the vehicle may generate a global path 8000on the basis of GPS position information of the origin and thedestination.

Also, the controller of the vehicle may reflect traffic informationbetween the position of the vehicle and the destination of the vehiclewhile generating the global path 8000.

As an example, the controller of the vehicle may set a path that allowsthe vehicle to travel from the position of the vehicle to thedestination of the vehicle in the shortest time as the global path 8000.

As another example, the controller of the vehicle may set a path thatallows the vehicle to travel from the current position of the vehicle tothe destination in the shortest distance as the global path 8000.

Also, the global path 8000 may not include a detailed path in units oflanes. In detail, the global path 8000 may not include detailed pathsthat allow the controller of the vehicle to control the vehicle tochange lanes.

In some embodiments, it will be appreciated that the global path 8000may include detailed paths in units of lanes.

FIG. 80 is a diagram illustrating a local path and a modified pathaccording to an embodiment.

Referring to FIG. 80 , the controller of the vehicle may generate alocal path 8100 along which the vehicle is to travel and then maydisplay the local path 8100 in a high-definition map 1420.

More specifically, the controller of the vehicle may generate a localpath 8100 related to at least a portion of the global path on the basisof sensor data for at least one object present in the field of view ofat least one sensor placed in the vehicle traveling along the globalpath.

However, the present invention is not limited thereto, and thecontroller of the vehicle may generate a local path 8100 on the basis ofthe sensor data and sharing data acquired from other devices.

More specifically, the controller of the vehicle may generate a localpath 8100 on the basis of sensor data for at least one object present inthe field of view of a sensor placed in the vehicle traveling along theglobal path and sensor data acquired from a sensor placed in otherdevices.

For example, when a vehicle located at a first point sets a second pointas a destination, a controller of the vehicle may generate a global paththat allows the vehicle to travel from the first point to the secondpoint and may generate a local path 8100 on the basis of sensor data andsharing data which are acquired while the vehicle is traveling along theglobal path.

Also, the local path 8100 may include a detailed path in units of lanes.In detail, the local path 8100 may include a detailed path that allowsthe controller of the vehicle to change lanes to travel on the nextlane.

Also, the local path 8100 may include an available movement region in avisible region of a driver.

Also, the local path 8100 may include a region including at least oneobject present in the field of view of a sensor placed in the vehicle.

Also, when the local path 8100 is generated based on sensor dataacquired while the vehicle is traveling and sharing data received fromother devices, the local path 8100 may include both of a regionincluding at least one object in the field of view of the sensor placedin the vehicle and a region including at least one object out of thefield of view of the sensor placed in the vehicle.

Also, the local path 8100 may include a modified path 8110.

In detail, when the controller of the vehicle detects an obstaclethreatening the vehicle on the global path or the local path of thevehicle, the controller of the vehicle may generate the modified path8110. In this case, the controller of the vehicle may set the modifiedpath 8110 as a local path along which the vehicle is to travel.

The modified path 8110 will be described in detail below (in Section5.2.2.2.2.).

5.2.2.2. Processing of Sharing Data and Generation of Path According toEmbodiment

According to an embodiment, a vehicle that has received sharing data maygenerate a path along which the vehicle is to travel on the basis of thesharing data and sensor data acquired from a sensor placed in thevehicle.

FIG. 81 is a flowchart illustrating a method of generating or modifying,by a vehicle, a path on the basis of sharing data according to anembodiment.

Referring to FIG. 81 , a controller of a vehicle may acquire a first setof point data through at least one sensor placed in the vehicle (S5063).

Also, the controller of the vehicle may determine first property data onthe basis of at least one subset of point data included in the first setof point data (S5064).

Also, the controller of the vehicle may generate a local path alongwhich the vehicle is to travel on the basis of at least a portion of thefirst set of point data, at least one subset of point data or the firstproperty data (S5065).

Also, a controller of a first device may acquire a second set of pointdata through at least one sensor placed in the first device (S5066).

Also, the controller of the first device may determine second propertydata on the basis of the second subset of point data included in thesecond set of point data (S5067).

Also, the controller of the first device may transmit sharing dataincluding the second property data to the vehicle (S5068).

Also, the vehicle may generate a modified path on the basis of thesecond property data and at least one of the first set of point data,the first property data, or the local path (S5069).

Hereinafter, the operations of the method in which the vehicle generatesor modifies the path on the basis of the sharing data according to anembodiment will be described in detail.

5.2.2.2.1. Generation and Sharing of Sensor Data and Sharing Data

FIG. 82 is a diagram showing a situation in which a first vehicletravels along a path generated based on sensor data and sharing dataaccording to an embodiment.

Referring to FIG. 82 , a controller of a first vehicle 141 may acquire afirst set of point data through a sensor placed in the first vehicle141, and a controller of an infrastructure device 700 may acquire asecond set of point data through a sensor placed in the infrastructuredevice 700 (S5063, S5066).

In this case, the first set of point data may include a first subset ofpoint data representing at least a portion of a building 500, and thecontroller of the first vehicle 141 may determine first property data onthe basis of the first subset of point data (S5064). In this case, thefirst set of point data may not include information regarding apedestrian 800 that is covered by the building 500 and thus placed outof the field of view of the sensor. Also, the first property data mayinclude center position information, size information, movementinformation, shape information, and the like of the building 500, butthe present invention is not limited thereto.

Also, the second set of point data may include a second subset of pointdata representing at least a portion of the pedestrian 800, and thecontroller of the infrastructure device 700 may determine secondproperty data on the basis of the second subset of point data (S5067).In this case, the second property data may include center positioninformation, size information, movement information, shape information,and the like of the pedestrian 800, but the present invention is notlimited thereto.

However, the present invention is not limited thereto, the second set ofpoint data may include a third subset of point data representing atleast a portion of the building 500, and the controller of theinfrastructure device 700 may determine third property data on the basisof the third subset of point data.

Also, the controller of the infrastructure device 700 may generatesharing data on the basis of the second property data and transmit thesharing data to the first vehicle 141 (S5068). In detail, theinfrastructure device 700 may transmit, to the first vehicle 141, secondproperty data generated based on a second subset of point datarepresenting the pedestrian 800 which is not included in the first setof point data. In this case, the second property data may includetracking information of the pedestrian 800 predicted according to themovement direction and movement speed of the pedestrian 800. In thiscase, the controller of the first vehicle 141 may compute theprobability of collision between the first vehicle 141 and thepedestrian 800 on the basis of the tracking information.

Also, the content of the sharing data may include the third propertydata. In some embodiments, the content of the sharing data may notinclude the third property data.

5.2.2.2.2. Generation and Modification of Local Path

The controller of the first vehicle 141 may generate a local path 8100on the basis of the sensor data acquired through the sensor placed inthe first vehicle 141 (S5065). In this case, the first vehicle 141 maygenerate the local path 8100 before receiving the sharing data from theinfrastructure device 700. In some embodiments, the first vehicle 141may generate the local path 8100 after receiving the sharing data.

As an example, the controller of the first vehicle 141 may generate thelocal path 8100 on the basis of the first property data. In detail, thecontroller of the first vehicle 141 may control the vehicle along aglobal path and may generate a local path 8100 on the basis of sensordata for an object present in the field of view of the sensor placed inthe vehicle. As a specific example, the controller of the first vehicle141 may generate a local path 8100 for preventing collision between thevehicle and the building 500 on the basis of the sensor data (e.g., afirst subset of point data or first property data) for the building 500.

It will be appreciated that in some embodiments, the controller of thefirst vehicle 141 may generate a local path on the basis of a first setof point data and a plurality of subsets of point data which areincluded in the sensor data.

Also, the controller of the first vehicle 141 may generate a modifiedpath 8110 on the basis of the sensor data and the sharing data (S5069).In detail, in order to avoid collision with an object not included inthe sensor data, the controller of the first vehicle 141 may generate amodified path 8110 on the basis of sensor data acquired from the sensorplaced in the first vehicle 141 and sharing data received from theinfrastructure device 700.

In this case, the content of the sharing data may include informationregarding the object not included in the first set of point data. Forexample, the content of the sharing data may include a second subset ofpoint data representing at least a portion of the pedestrian 800 notincluded in the first set of point data or second property data of thesecond subset of point data, but the present invention is not limitedthereto.

Also, the controller of the first vehicle 141 may determine whether togenerate the modified path 8110 on the basis of the second property databefore generating the modified path 8110.

As an example, when the local path 8100 includes at least a portion of apredetermined region where the pedestrian 800 is located, the controllerof the first vehicle 141 may generate the modified path 8110. In otherwords, when the local path 8100 and the predetermined region where thepedestrian 800 is located partially overlap each other, the controllerof the first vehicle 141 may generate the modified path 8110. In thiscase, the predetermined region may be preset by the controller of thefirst vehicle 141. However, the present invention is not limitedthereto, and the predetermined region may be set based on the speed ofthe first vehicle 141, the distance to the pedestrian, or the like.Also, the modified path 8110 may not overlap the predetermined regionwhere the pedestrian 800 is located. It will be appreciated that in someembodiments, the modified path 8110 may partially overlap thepredetermined region where the pedestrian 800 is located.

As another example, the controller of the first vehicle 141 may computethe probability of collision between the first vehicle 141 and thepedestrian 800 on the basis of second property data including movementinformation of the pedestrian 800 not included in the first set of pointdata and may determine whether to generate the modified path 8110according to the computed probability.

More specifically, the controller of the first vehicle 141 may determinewhether to modify the path of the vehicle on the basis of theprobability of movement of the first vehicle 141 predicted based on thelocal path 8100 of the first vehicle and the probability of movement ofthe pedestrian 800 predicted based on the second property data.

As a specific example, the controller of the first vehicle 141 maydetermine whether to generate the modified path 8110 on the basis of acollision probability map, which is generated based on the local path8100 and the second property data and along with the movement of thefirst vehicle 141 and the pedestrian 800.

FIG. 83 is a diagram illustrating a method of generating a modified pathon the basis of a collision probability map generated by a controller ofa first vehicle according to an embodiment.

Referring to FIG. 83 , a controller of a vehicle may generate acollision probability map that represents the probability of movement ofa pedestrian and the probability of movement of the vehicle travelingalong a local path 8100 over time. In this case, when a region 8200having a high probability of collision between the vehicle and thepedestrian is shown in the collision probability map while the vehicleis traveling along the local path 8100, the controller of the vehiclemay determine to generate the modified path 8110 so as to avoidcollision and may generate the modified path 8110.

However, the present invention is not limited thereto, and thecontroller of the vehicle may determine whether to generate the modifiedpath according to whether a blind spot is present in sensor dataacquired from a sensor placed in the vehicle.

In detail, when a blind spot is detected in the scheme described inSection 4.2.5., a controller of a vehicle traveling along a local pathmay generate a modified path to avoid possible dangers due to thepresence of the blind spot.

For example, when a blind spot is detected, the controller of thevehicle may generate a modified path to decelerate the vehicle or changelanes, but the present invention is not limited thereto.

5.2.2.2.3. Various Examples of Modified Path

Also, a controller of a vehicle may generate an optimal modified path toavoid collision between the vehicle and other objects.

FIG. 84 is a diagram illustrating various examples of a modified pathaccording to an embodiment.

Referring to FIG. 84 , a controller of a second vehicle 142 travelingalong a global path and a local path may generate at least one modifiedpath in order to avoid a pedestrian 800 on the basis of the movementspeed, movement direction, position, and the like of the second vehicle142.

For example, the at least one modified path may include a first modifiedpath 8111 for stopping the second vehicle 142 and a second modified path8112 for changing at least a portion of the local path, but the presentinvention is not limited thereto.

In detail, the controller of the second vehicle 142 may receiveinformation (e.g., property data) regarding the pedestrian 800 which isnot included in sensor data acquired from a sensor placed in the secondvehicle 142 and which is included in sharing data received from otherdevices. In this case, the controller of the second vehicle 142 maygenerate a first modified path 8111 for stopping the second vehicle 142in order to prevent collision between the second vehicle 142 and thepedestrian 800. In this case, the first modified path 8111 may begenerated to stop the second vehicle 142 in a predetermined time or stopthe second vehicle for a predetermined time.

Also, the controller of the second vehicle 142 may generate a secondmodified path 8112 which allows the second vehicle 142 to avoid thepedestrian 800 by changing at least a portion of the local path of thesecond vehicle 142 so as to prevent collision between the second vehicle142 and the pedestrian 800.

However, the present invention is not limited thereto, and thecontroller of the second vehicle 142 may generate a modified path bychanging at least some of the position, speed, and direction of thesecond vehicle 142 which are included in the local path.

When the controller of the vehicle generates a modified path includingthe position, speed, or direction of the vehicle, the controller of thevehicle may set the modified path as a local path and may control thevehicle on the basis of the local path.

Also, the modified path may include a path obtained by modifying atleast a portion of the global path. In detail, when informationindicating that a specific event has occurred on the global path of thevehicle, the controller of the vehicle may generate a modified pathreflecting the information indicating that the event has occurred andmay set the modified path as a new global path.

As an example, the controller of the vehicle may control the vehiclealong a first global path which is generated based on the currentposition of the vehicle and the position of the destination of thevehicle. In this case, when the controller of the vehicle receivessharing data including information related to a traffic event that hasoccurred at a specific time related to the first global path fromanother device, the vehicle may generate a modified path such that thevehicle can avoid the region where the traffic event has occurred. Inthis case, the controller of the vehicle may set the modified path as asecond global path and control the vehicle along the second global path.

As described above, the modified path may refer to a path obtained bymodifying at least a portion of the global path or the local path.However, the present invention is not limited thereto, and the modifiedpath may refer to a path for suddenly stopping the vehicle. Also, thecontroller of the vehicle may set the modified path as a new global pathfor the vehicle or a new local path for the vehicle.

5.2.3. Case in which Information Related to Traffic Event is Included inSharing Data

The content of the sharing data according to an embodiment may includeinformation related to a traffic event such as a traffic accident. Inthis case, the traffic event-related information may refer toinformation indicating that at least one object is associated with atraffic event. However, the present invention is not limited thereto,and the traffic event-related information may refer to a message thatrequests information regarding the traffic event or the like.

In this case, a device which has received the traffic event-relatedinformation may display data (e.g., an event occurrence region) includedin the traffic event-related information in a high-definition map. Forexample, in order to notify an occupant of information related to thetraffic event, a controller of a vehicle that has received the trafficevent-related information may display a region where the traffic eventhas occurred in a high-definition map.

Also, the device which has received the traffic event-relatedinformation may change property data (e.g., class information) ofobjects related to the event using the traffic event-relatedinformation.

As a specific example, a controller of a first vehicle may determinethat class information of a second vehicle included in sensor dataacquired through at least one sensor included in the first vehicle is“vehicle.” In this case, when the second vehicle is an object related toa traffic event, the controller of the first vehicle may receiveinformation indicating that the second vehicle is related to the trafficevent from the second vehicle. In this case, the controller of the firstvehicle may change class information of the second vehicle to“accident,” “accident vehicle,” “accident site,” “accident point,” orthe like, but the present invention is not limited thereto.

Also, the controller of the first vehicle may control the first vehicleon the basis of the changed class information of the second vehicle. Forexample, the controller of the first vehicle may generate a local pathnot including a region related to the second vehicle related to thetraffic event, but the present invention is not limited thereto.

6. Various Applications Using Sensor Data and Sharing Data

The method of selectively sharing and processing the sensor data and thesharing data according to the above embodiment may be used in variousapplications.

As an example, the method of selectively sharing and processing thesensor data and the sharing data may be used for a black box (a dashcam). In this case, a vehicle including a black box using a LiDAR maystore a set of point data acquired using the LiDAR in a memory of theblack box or a memory included in the vehicle. However, as describedabove, in order to solve a storage capacity issue of a memory and aprivacy invasion issue caused by intensity information of an objectacquired from a LiDAR, the controller of the vehicle may selectivelystore the set of point data. For example, the controller of the vehiclemay store a set of point data other than the intensity information ofthe object, but the present invention is not limited thereto. Thecontroller of the vehicle may generate and store privacy protection dataobtained by partially processing a subset of point data representing atleast a portion of the object.

Also, when a vehicle is related to a traffic event such as trafficaccident, the vehicle may receive sharing data including privacyprotection data according to class information of the object related tothe traffic event from a nearby device or may selectively receive onlydata related to a movable object as described above. In this case, thecontroller of the vehicle may reconfigure the traffic event on the basisof the sharing data.

Also, as described above, a vehicle located near the region where thetraffic event has occurred may receive a request for sensor data relatedto the traffic event from a server, and a controller of the vehicle maytransmit sharing data related to the traffic event to the server inresponse to the request. In this case, the server may reconfigure thetraffic event on the basis of the sharing data related to the trafficevent.

Also, as described above, a device which has received the sharing datarelated to the traffic event may match a plurality of pieces of data byaligning the coordinate systems of the sharing data and the sensor datawith a signal coordinate system. In this case, the device mayreconfigure the traffic event by listing, in chronological order, sensordata and sharing data which are acquired for a predetermined time beforeand after the traffic event.

As another example, as described above, the method of selectivelysharing and processing the sensor data and the sharing data may be usedto detect a blind spot which refers to a region where information cannotbe acquired from a sensor placed in the vehicle. In detail, in order toacquire information regarding an object that is placed in the field ofview of a sensor placed in a vehicle and that is covered by anotherobject and thus is not included in sensor data, a controller of thevehicle may receive sharing data including the information regarding theobject not included in the sensor data from other devices.

In this case, the device which has transmitted the sharing data to thevehicle may selectively generate the content of the sharing data on thebasis of class information of an object included in the sensor data.Also, the device may selectively generate the content of the sharingdata according to whether an event related to the vehicle has occurred.Also, a vehicle that has received the sharing data may match data on anobject located in the blind spot and sensor data acquired by the sensorplaced in the vehicle through coordinate system alignment. In this case,the controller of the vehicle may control the vehicle on the basis ofthe matched sensor data and sharing data.

As still another example, the method of selectively sharing andprocessing the sensor data and the sharing data may be used to detect anavailable parking space of a vehicle as described above. As a specificexample, when a vehicle enters a parking lot, the vehicle may receiveinformation regarding the available parking space from an infrastructuredevice placed in the parking lot. In this case, the controller of thevehicle may autonomously park the vehicle in the available parking spaceusing an autonomous parking system and a system for communication withthe infrastructure device.

Section 6 illustrates that the above descriptions in Sections 1 to 5 areapplicable to some applications, and it will be appreciated that thedescriptions in Sections 1 to 5 except for the description in Section 6are also applicable to the applications. Also, it will be appreciatedthat the above descriptions in Sections 1 to 5 are applicable toapplications (e.g., a traffic control system and any mode oftransportation (drone, ship, train, etc.) other than vehicles) otherthan the applications described in Section 6.

The method according to an embodiment may be implemented in the form ofprogram instructions executable by a variety of computer means and maybe recorded on a computer-readable medium. The computer-readable mediummay include program instructions, data files, data structures, and thelike alone or in combination. The program instructions recorded on themedium may be designed and configured specifically for an embodiment ormay be publicly known and usable by those who are skilled in the fieldof computer software. Examples of the computer-readable medium include amagnetic medium, such as a hard disk, a floppy disk, and a magnetictape, an optical medium, such as a compact disc read-only memory(CD-ROM), a digital versatile disc (DVD), etc., a magneto-optical mediumsuch as a floptical disk, and a hardware device specially configured tostore and perform program instructions, for example, a read-only memory(ROM), a random access memory (RAM), a flash memory, etc. Examples ofthe computer instructions include not only machine language codegenerated by a compiler, but also high-level language code executable bya computer using an interpreter or the like. The hardware device may beconfigured to operate as one or more software modules in order toperform the operations of an embodiment, and vice versa.

Although the present disclosure has been described with reference tospecific embodiments and drawings, it will be appreciated that variousmodifications and changes can be made from the disclosure by thoseskilled in the art. For example, suitable results may be achieved if thedescribed techniques are performed in a different order and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner and/or replaced or supplemented by othercomponents or their equivalents.

Therefore, other implementations, embodiments, and equivalents arewithin the scope of the following claims.

What is claimed is:
 1. A method for sharing and using sensor data in aserver, the method comprising: establishing a wireless communicationlink between the server and a plurality of devices via a communicationnetwork, wherein at least one of the plurality of devices comprises asensor configured to obtain a set of point data corresponding to one ormore objects, and wherein the plurality of devices are configured toobtain: object location information, indicating a location of the one ormore objects, from the set of point data, sharing data from the set ofpoint data, wherein the sharing data includes at least one of aplurality of sets of property data of the one or more objects, and eventinformation comprising event occurrence information indicating that anevent occurred, event time information indicating a time of the event,and event location information indicating a location of the event;receiving, via the communication network, device location information,indicating a location of at least one of the plurality of devices, andthe object location information from at least one of the plurality ofdevices; receiving, via the communication network, the event informationfrom at least one of the plurality of devices; determining an occurrenceof the event relating to a first device of the plurality of the devicesor at least one object of the one or more objects based on the eventoccurrence information; determining a time of the event relating to thefirst device based on the event time information; determining a locationof the event relating to the first device based on the event locationinformation; receiving, via the communication network, the sharing datafrom a second device of the plurality of devices, wherein the seconddevice is configured to obtain a first set of property data of the atleast one object relating to the event; generating a first determinationresult indicating whether or not the second device is within a firstrange from the location of the event based on the determined location ofthe event; generating a second determination result based on the sharingdata received from the second device, the second determination resultindicating whether or not the sharing data includes the first set ofproperty data of the at least one object relating to the event;selectively transmitting, via the communication network, a first messageor a second message to the second device at least partially based on atleast one of the first determination result or the second determinationresult, wherein the first message comprises the determined time of theevent, and is configured to request a first set of point data obtainedfrom the second device, wherein the second message comprises thedetermined location of the event, and is configured to notify the seconddevice of the event, wherein the first message is transmitted to thesecond device, in response to the first determination result indicatingthat the second device is within the first range and the seconddetermination result indicating that the sharing data received from thesecond device includes the first set of property data, and wherein thesecond message is transmitted to the second device, in response to thefirst determination result indicating that the second device is notwithin the first range or the second determination result indicatingthat the sharing data received from the second device does not includethe first set of property data; and in response to the first messagebeing transmitted to the second device, receiving, via the communicationnetwork, from the second device, a second set of point data obtainedwithin a first time period including the time of the event.
 2. Themethod of claim 1, wherein the event includes at least one of atraffic-event related to at least one of accident related to the firstdevice or accident related to another device around the first device, anenvironment event related to environment around the first device, or aregulatory event related to regulatory about the first device or anotherdevice around the first device.
 3. The method of claim 1, wherein one ormore of the plurality of devices comprise at least one of a movingobject, a mobile device, or an infrastructure device.
 4. The method ofclaim 1, wherein the plurality of devices include at least one sensor,and wherein the at least one sensor includes at least one of a LiDAR, acamera, a radar, or an ultrasonic sensor.
 5. A non-transitorycomputer-readable recording medium storing instructions thereon, whenexecuted by a processor, configured to perform the method of claim
 1. 6.The method of claim 1, further comprising: generating a thirddetermination result indicating whether or not a third device of theplurality of devices is located within a second range, wherein a thirdmessage is transmitted to the third device, in response to the thirddetermination result indicating that the third device is located withinthe second range, and wherein the third message comprises the determinedlocation of the event, and is configured to notify the third device ofthe event.
 7. The method of claim 1, wherein the property data for theobject related to the event includes at least one of class informationof the object, center position information of the object, sizeinformation of the object, movement information of the object, shapeinformation of the object, or identification information of the object.